Mobile App Performance SDK - Review Guidelines
We want to make sure you have the Mobile Application Performance (MAP) SDK configured properly, and that you’re getting the most out of it. The document is designed to help App owners and developers analyze, optimize and validate results with MAP SDK while making sure that your settings are configured correctly.
It is great that you are using MAP SDK with your native apps to enhance performance. However, this is far from a one-time activity and would only add incremental value if you continue to maintain the configuration/apps launched from time to time.
We recommend a continuous improvement-based journey involving the three main steps shared below to maximize benefits from MAP SDK. (focused on not just a one-time fix, but followed by iterative improvement cycles to maximize performance and end-user experience)
- Monitor – Measure application behavior (After release) in real-world conditions and understand how traffic and experience could be optimized. Review our Configuration Guide to understand the different reports/insights available.
- Optimize – Make improvements (Including automatic optimizations applied by MAP SDK) to the application or delivery network/infrastructure to improve performance/user experience.
- Validate – Measure and validate the application behavior (against real-world conditions) to confirm improvement in user experience based on recommendations made by optimizations applied
Note: Please read the Mobile Application Performance (MAP) SDK Configuration Guide before using this document.
In the following sections you will learn how to verify configurations of the following MAP SDK features;
- SureRoute for Cellular (SR4C)
- Universal Cache (UC)
- Adaptive Image Compression (AIC)
- Image Manager (IM)
- Adaptive Network Optimization (ANO)
2. SureRoute for Cellular (SR4C)
SureRoute for Cellular performs two DNS lookups: one using the device’s default resolver, the other using a DNS server such as OpenDNS or Akamai’s Answer-x. Issuing two requests improves the chances of a successful request through unreliable network connections or bottlenecks, and increases the chances of retrieving an object from the Edge cache. It’s basically a race to see which returns bytes faster.
Validate URL by Regex:
- (Validation) URL entered should have format: <scheme>://<hostname>/<path>, where highlighted portion is mandatory.
- (Validation) No regular expressions allowed in scheme and hostname. The idea is to avoid someone from entering patterns such as “.*” or http://.*
- (Validation) Scheme MUST be either http/https. We do not allow any other protocols.
- (Validation) There MUST be a slash “/” after hostname. So, the simplest form of input would look like: “http://www.abc.com/”. This is needed because a URL without “/” ends up in redirection.
- (Validation) Regex use is allowed ONLY in <path> i.e. first slash “/” after <hostname>.
- (Validation) Use of backslashes (escaping). We want to enforce the use of “\” in special cases such as square brackets (“[”, ”]” MUST be prefixed as “\[“, “\]”)
- (Indication) “?” character when not being part of Regex expression must be prefixed with a backslash (“\?”. For ex – at the beginning of query string)
- (Indication) Examples related to above validations. The idea is to let customer know how SDK is going to interpret a given regex.
Testing Regex: https://regex101.com/
Enter your matches and test them against the top URLs in the app to see if they match.
In the above image(Image 1) we check for Regular Expression for sample.hostname.com\.*\.jpg. We checked across two sample URL’s to verify if they match the Regex pattern and thus confirming the applied Regex is fine.
Validate Stickiness interval: Default value should be set to 60 seconds, unless the same is changed with a clear intention to reduce stickiness.
The stickiness interval controls how long the client will reuse race results. The lower the value, the more races will be run (which is not so effective, especially if the value is less than 60). The higher the value, fewer races will be run (Not optimal either if the value is above 60 seconds). Risks are introduced if the value is too high (above 60 seconds) because the network/server conditions may vary and the best performing server may change.
Validate that all hostnames for all SR4C matches are indeed live with ION. Do not run races on objects that are not on ION. Only ION has the protections in place to prevent both primary and secondary requests from being sent to the origin. Without this protection, both requests could be sent to the origin, which is a traffic amplification. An easy way to verify this is to look for ANO profiles in the ANO report. If there are profiles for this hostname then the hostname is on Ion. If there aren't any profiles then the hostname may not be on Ion. For a more authoritative, answer, find the hostname in Akamai Property Manager in Luna and ensure that the configuration type for the property is Ion.
In the above image(Image 2), we checked ANO reports and it shows which hostnames are on ION and not on ION. Check for a number of profiles for a hostname, if there are no profiles then the hostname may not be on ION.
SR4C report: The SureRoute for Cellular Report tells how often you’re racing and how often a race benefits other objects. In a simpler way, how many races are running for a hostname and how many URLs on that hostname are benefiting from the races. Here’s an example:
First, you want to confirm is that you’re only initiating SR4C races on cacheable content. Racing on uncacheable has no positive benefit. The SDK will issue two requests for the same uncacheable object, at the Edge, only the primary request will be forwarded to the origin. The secondary request will be terminated. This is so we do not unintentionally amplify the number of requests to the origin. This would be especially bad in the event of a cart or purchase request.
Second, check to look at the hostnames and confirm that you do see races on hostnames that primarily serve images (often these have the format of img.hostname.com or i.hostname.com or similar).
Third, you can click on a hostname with races to see what kind of URLs and file extensions you’re racing on:
In the above case(Image 4), you can see that there are many .jpg file extensions which is great. Images are usually cacheable, and therefore our matches should be for image file extensions and if we wrote our matches correctly, we will see image file extensions in our race URLs list. Fourth, check out the benefitting URLs list. You should also see cacheable file extensions in this list. And in the example below we do see more .jpg’s:
Fifth, you want to confirm that hostnames with uncacheable content such as api.hostname.com do not have races. It’s true that API calls can be cacheable. But it’s also true that most times API calls are not cached on the Edge and unless you know for sure, you should make sure you are not racing for uncacheable content. A quick scan of the top hostnames by hits shows that many hostnames do not have races which are good.
If you’re not sure, one way to verify if races are occurring on a hostname with cacheable objects or not is to look at the Offload report. Note that Universal Cache (UC) must be enabled for the report to help you. However, if UC is enabled, then you can compare the hostnames that have races to hostnames with cache hits.
In the above example(Image 6 and 7), we see that hostnames with races also have cache hits, which is a good sign. Also note that the top hostname has very few cache hits, and also has no SR4C races, another good sign.
Next, you can explore further in the Offload report to see what the top cacheable objects look like. Start with the objects that are served from the cache (since it’s the default). These objects are cacheable which is good, and they should have a match in your SR4C config, but they also are served from cache frequently so they may not generate a lot of races or benefit much from SR4C. Still, it’s worth adding the match.
In the example above you can see that most of the most popular cacheable images are png files. A quick scan of our SR4C match shows that there is no match for .png so there’s an optimization. We can add a URL for png regex to match the pattern.
Next, you can check for the top cacheable objects that were NOT served from the cache. This is the sweet spot since they are cacheable objects, but frequently generate network requests. In this case, the objects are all .jpg images.
If you compare the list of cacheable objects, that are not being served from cache to the list of the most popular SR4C objects, you should see overlap and in fact, we do. Although, we also see almost as many race URL hits as we do total objects requests, which we’ll discuss below.
Next, you can look for objects that are not served from the cache because they’re uncacheable. You do not want to see these URLs in your SR4C race list for the reasons mentioned above. (The objects are uncacheable and will never benefit.)
Another way to validate cacheability is to look at the top race URLs and make sure they're cacheable. You can do this by looking at cache-control headers with your HTTP tool.
Example in Image 13 shows: Google Chrome: Inspect-> Network:
Next, let’s examine the race and benefitting rates in SR4C report:
We alluded this above...but the race rate is too high. This is not an ideal setup. The ideal setup will have a low race rate and a high benefitting rate. Races are expensive since they issue two requests for the same object. Therefore you want to minimize the race rate and maximize the benefit rate, which indicates that many network requests took the optimal path and benefitted from a single race.
The target race rate and the benefitting rate will depend on the app. The ideal race rate for an app that consistently shows 20 images per-app view, would be 5% and the ideal benefitting rate for that app would be 95%. This would indicate that the first image triggered a race and the subsequent 19 images benefitted from that race. These would be unusually remarkable results.
The real world is more complicated. Not all app views have the same number of images. Some have none and some have an infinite scroll. Also keep in mind that the universal cache offload will lower the number of network requests, which will reduce both the race and benefitting rates.
For a hostname that carries almost exclusively cacheable images, you typically want to see a race rate of about 5% and you would want the benefitting rate to be as high as possible. Benefitting rates of 50% or more can be achieved.
When you see race rates above 10% you should look to optimize your matches to reduce the number of races. Likewise, if you’re not achieving a benefitting rate that is at least 4 to 5 times higher than the race rate, you’re not reusing your race results. Remember you want to race 1 time and benefit many times. For example, if your race rate is 5% and your benefitting rate is 25% then on average, 5 network requests are benefitting from every race. That’s a good result.
3. Universal Cache
Offload Reports: We're trying to optimize for the greatest benefit. So you can start with hostnames carrying the greatest number of hits. For hostnames that primarily serve images, we hope to achieve 20% or greater Cache Hit Rate. It’s important to remember that the cache is on the device and only benefits when the same user requests the same object multiple times.
If the hostname is an API hostname it is common to see lower cache hit rate (Eg: Objects like XML, JSON).
Note: Offloading API’s as and where possible, certainly has a large impact on App performance. If the hit rates are low, one can certainly use this as an opportunity to improve the same.
Filter for objects for NOT Served from Cache (NOT Cacheable) as shown in the below image. If you see png, jpg, and other cacheable content types under “NOT Cacheable” then you need to find out why those are not being cached.
If an object size is larger (>3MB) MAP SDK won’t cache it, to avoid exhausting end-user device storage. However, usually bad cache control headers, lack of timestamp, etc lead to loss of offload. You will need to investigate such cases and you can do this with an HTTP inspector.
If you see file extensions that are typically uncacheable (XML, JSON), then you should try to explore those objects/ URLs to verify if they are potentially cacheable.
You will also find objects that are just not requested more than once per user.
Example: in a news app, the user may read a news article and may not read the same article again.
Thereby a URL is requested only once, and it's cacheable, but is not requested again, or the TTL is very low so we rarely see cache hits.
During your offload analysis, it is always more effective to consider the top 10 objects first and verify If caching those objects would increase the cache hit rate by more than a couple of % points, or if it would offload millions of requests. In such cases, it probably makes sense to try to optimize and offload them by adding or increasing TTL values.
You should also consider using pre-caching as an optimization to increase offload.
Note: If you're going to enable pre-caching (which is highly recommended) the TTL should be at least as long as the typical session (at least 10 minutes and ideally longer).
4. Adaptive Image Compression (AIC)
Check if Network Awareness (NA) is turned ON within MAP SDK configuration and your ION configuration for the property must have AIC enabled as well.
AIC depends on Network Awareness callbacks from the edge to decide on the quality of the image to request/serve. Also check if AIC is enabled, Image Manager (IM) should be disabled and vice versa.
5. Adaptive Network Optimization (ANO)
This feature is enabled by default by the SDK. Any hostnames on Ion will benefit. This is true, even for 3rd party hostnames that happen to be on Ion. The feature leverages algorithms to manage the Transport layer protocols profiles which can range from conservative to super-aggressive to maximize bandwidth and to reduce retransmissions.
Check the ANO hits ratio under MAP SDK ANO reports. If you don't see any ANO hits or profiles in the charts and you know for sure that ANO is enabled, then you're probably looking at a hostname that is not on ION.
Note that this is also a good check for SR4C. If you find a hostname without Ion, you want to make sure that SR4C is not enabled on it. Also, note that sometimes 3P hostnames will have profiles. This is good and it speaks to the popularity of our network. What it means is that this customer is benefitting by the fact that the 3P company's hostname is also on Ion.