You wouldn’t look out your window to determine the weather on the other side of the world, and yet that’s more or less how people test their websites. The fact is, a person accessing your site from a laptop in China, or a mobile device in Brazil, might have a very different user experience than you do in your office.
How long does it take for your page to load? How long do users have to wait until they can interact with it? Studies show most people will wait about two seconds before going elsewhere. Think about that for a second. Or make that two seconds, because that’s as long as you’ve got to make an impression. Speed isn’t everything, but it’s most of it.
This is why it’s important to test how fast your page loads from different devices and geographical locations. Not only you can get underneath that initial two seconds threshold, but also to continually improve your users’ experience, wherever they are.
Selecting a Site for Testing
To demonstrate my point, let’s evaluate a website over two geolocations, using different Internet connections.
First, I need to select a site and found this blog (or should I say award) about good designs and picked one that called my attention. The My Own Bike site looks great, it's not using a Content Delivery Network (CDN), and is hosted from Germany.
Second, let’s set up a plausible story line: My Own Bike wants to sell globally. I created this market analysis using Akamai’s State of the Internet Q3 2016 report and the Top 10 Countries with Most Bicycles per Capita article.
|Location||Desktop (Avg Mbps)||Mobile (Avg Mbps)||Bicycles (% population)|
|United States||16.3||7.5||100 million / 32%|
|China||5.7||8.9||500 million / 37%|
|Japan||18.0||11.6||72 millions / 60%|
|Germany||13.7||13.1||60 millions / 75%|
|Netherlands||17.3||11.5||16.5 millions / 99.1%|
|Denmark||16.6||12.4||4.5 millions / 80%|
Looks like United States and China are good bicycles market with room to grow, so let’s use those countries for our market expansions.
Third, I need to evaluate how the site performs in those geos. I can't travel to China and test from there (I tried but my manager did not approve the budget). My best, and free option, was to use WebPageTest from China, but since the China node wasn't available I used the closest geolocation available: South Korea. For the U.S., I used San Jose in California.
My test used the download numbers from above and I set to half of that for upload. In desktop, I used 50ms latency with 2% packet lost. For mobile, I used 200ms and 5%, as that's around what I normally see. I ran the test seven times and chose the median for the results in this report.
So the big question is, how is the user's experience impacted by the context? Let’s run some tests.
Testing Mobile Speed in the US
Would shoppers in the U.S. using a mobile connection speed see any difference with shoppers using a desktop? Take a look at the following filmstrip.
The image speaks for itself. Document complete is 11.6s for desktop and 27.7s for mobile. In fact, the desktop is visually complete by the time mobile started rendering. As you can see, the site is miserably slow from either connection, and I’ll get into why shortly, but even in the same geo, mobile takes about 240% the time.
Testing Geographical Location
There are three main variables in remote tests: distance to the origin, download/upload connection speed, and latency. To test this, let’s add China to the mix, and keep the mobile vs. desktop pairing as well.
I’m using a different graph this time because the filmstrip was just too wide apart to be useful.
Here are some key takeaways:
- The faster U.S. desktop speed (green) leads to faster visual progress and load times than a desktop in China (red).
- Conversely, mobile users in China (blue) get a better experience than those in the U.S. (yellow), thanks to faster average mobile speed.
- The most interesting result is that China’s desktop experience is faster than the mobile one over the same hardware, even though mobile is 3.2 Mbps faster. The reason is 150ms of added latency and an increase of 3% in package loss.
Analyzing the waterfall graph we can see how the connection and download times are affecting the load time. This image is a good example:
Flattening the Earth with a CDN
A content delivery network (CDN) like Akamai positions content at the edge, close to where it’s being requested. Akamai’s intelligent platform also reduces byte size without impacting quality, which further speeds up loading time.
Let’s see what happens when we add a CDN with compression and image optimization capability. Take a look at the new pink and orange lines on the far left of the graph. The difference is astonishing.
Every metric improves and the earth “flattens”. Users in China and U.S. have virtually the same experience. The next point will review the image optimizations in more detail.
Images Are Often the Problem
On average, 63% of page weight is images. Sites are image hoarders for a reason: it drives conversion (learn more with this Understanding the Image Challenge video). If you scroll back up and look at the waterfall graph, notice how much time is spent on .jpg and .png files.
Akamai’s image optimization solution transforms the images to the browser-preferred format (e.g. webp for Chrome) and compress the images intelligently, without perceived quality lost. There is a small savings in bytes by doing these as shown below (I'm using Piez to get this data).
The image transformation is also visible in WebPageTest where the new image is identified as webp:
The size difference might not look like much, but it could result in fewer roundtrips, especially over HTTP/2.
There are many factors that impact your user’s experience, such as location, network condition, latency, origin location, device, and almost anything else you can think of. Combined together, they can conspire to make your perfectly designed site fall on the wrong side of user expectations.
CDNs such as Akamai that can adjust the content to the situation and device (convert images to device-specific format, or compress using Brotli to browsers that support it) will help you level the user experience across the globe, and increase the percentage of users receiving a good experience.