Blog

Aggregate Waterfall

Applying Waterfall Visualizations to Real User Monitoring Data

October 10, 2018 · by Simon Hearne ·

Waterfall charts have been a staple of web performance analysis from the very beginnings of our industry in 2003, thanks to IBM Page Detailer and continued development on FireBug and of course WebPageTest.org. These visualizations are our default tools to understand the structure of our pages and have been fundamental to our understanding of web performance.

webpage test waterfall

Waterfall charts have one major drawback for performance analysis: they only show a single point-in-time, often from a synthetic test agent. This is a particularly important issue for resource performance, as many third-party resources behave differently for synthetic tests compared to real users, specifically tag managers and advertising networks which use cookies to determine their behavior. Analyzing the impact of third-party resources on page load becomes almost impossible with dynamically added resources.

While there is undoubtedly value in using synthetic tests for analyzing web performance, Real User Monitoring (RUM) is the gold-standard for performance analytics. Solutions such as Akamai mPulse collect huge amounts of data from real users including their interactions with your applications in real conditions.

A relatively new addition to the RUM toolkit is Resource Timing, the ability to measure the response times of individual resources on your pages. mPulse was one of the first RUM solutions to collect and present this data.

mPulse has provided a beacon waterfall dashboard since we’ve collected the data, and we continue to work on improvements. The individual beacon waterfall is a great tool to investigate specific issues, but does not give an aggregate view of resource loads across multiple sessions. This limits the possibility of identifying trends or anomalies across multiple page loads.

mPulse waterfall

Enter Aggregate Resource Waterfall, one of the latest widgets to come to mPulse. This new data visualization takes the traditional web page waterfall concept and applies it to the huge amount of data available from RUM. Rather than present absolute values for a single test, the aggregate resource waterfall presents distributions of resource load times relative to page construction timers.

aggregate waterfall

Aggregate Resource Waterfalls use area charts to represent how often resources were loading at any time in the page load. The x-axis shows the percentage of page load completion relative to the chosen timer, instead of the absolute time in seconds. This normalization of resource performance against variations in the overall page load allows us to present resource timing data aggregated across all user sessions, while also reducing noise in the data from variable connection speeds, device types and user location. The shape and position of the distributions now tell us a lot more about the resource performance and loading patterns, potentially across millions of page loads!

aggregate waterfall annotated

Resources are initially grouped by host in order to condense the vertical size of the waterfall, selecting a host group expands and focuses on that host for further analysis.

Resources can be filtered by type, so if your focus is on third-party JavaScript just double-click the JavaScript filter to narrow the view.

Aggregate Resource Waterfall respects any filters you set in mPulse, allowing direct comparison of waterfalls by browser, device type or even whether the user was detected as being a bot or not!

This has already proved to be valuable for some clients with early access to the widget. One client used it to validate that critical fonts were being pre-loaded as expected for all users, another identified a potential 16% loss in analytics data due to third-party scripts loading out-of-sequence in some real user scenarios.

Aggregate Resource Waterfall is now available to all mPulse Enterprise customers, just contact your account representative to get started. If you are not currently an mPulse customer, sign up for your mPulse Lite free trial here.