Please note that prior to March 30th, 2019, Splunk Synthetic Monitoring collected a metric called First Interactive Time. This has been replaced with Time to Interactive. We have updated the name and the algorithm to collect this metric for improved accuracy and to be consistent with tools like Google Lighthouse. You may see timing differences for Time to Interactive due to this update.
There are two metrics that Splunk Synthetic Monitoring offers that measure interactivity: Time to Interactive and First CPU Idle.
Time to Interactive
Time to Interactive indicates the amount of time until a page is first expected to be usable and will respond to user input quickly. It can be a great metric to determine when users can reasonably expect to interact with a page, but it is important to understand how Time to Interactive is calculated to know if it is the right metric to monitor for your application.
Unlike performance and paint timings, there is no browser-supported API for interactive timings, and there is no standard definition for when a page is considered "interactive". We tested multiple approaches for monitoring interactivity and settled on a solution for Time to Interactive time that yields consistent results without affecting check timings.*
*While there are multiple definitions of the metrics from different tools, Splunk Synthetic Monitoring's definition of Time to Interactive is consistent with Google Lighthouse.
To calculate Time to Interactive, Splunk Synthetic Monitoring creates a PerformanceObserver to collect long task entries that can block user interactivity while your page is loading. After the page load is complete, we analyze the long task entries as well as network activity following the steps below:
- Find interactive windows where there is no long task that takes more than 50 ms* to complete, and no more than two successful in-flight non-GET requests.
*Tasks shorter than 50 ms won’t be perceived by users so they are ignored.
- Starting at First Contentful Paint, begin searching for a 5 second interactive (quiet) window.
- Time to Interactive is the end of the last long task before the interactive window.
While Splunk Synthetic Monitoring's approach to calculate Time to Interactive works for most applications, you may want a measurement of interactivity that is customized for you. To do that, you can use the User Timing API to log an event when a specific element appears or when a certain resource has loaded, and Splunk Synthetic Monitoring will monitor it. User timings appear on run result pages and in the Performance KPIs report.
First CPU Idle
First CPU Idle indicates when a page is minimally interactive. It is defined as the start of the first interactive window or DOM Content Loaded, whichever is first. A page is considered minimally interactive when:
- Most, but not all, UI elements on the screen are interactive.
- The page responds, on average, to most user input in a reasonable amount of time.
This metric can stand on its own, but is also important because it is a factor in the Performance Score. The industry standards for what “interactivity” means is constantly evolving, but as it stands now, First CPU Idle has a more forgiving definition for interactivity when compared to Time to Interactive.
To calculate First CPU Idle, Splunk Synthetic Monitoring follows the same basic approach we use for Time to Interactive. Splunk Synthetic Monitoring creates a PerfromanceObserver to collect long task entries that block user interactivity.
However, the interactive window for First CPU Idle is defined differently from Time to Interactive. Interactivity is determined by task clusters (periods of time where there may be many long tasks) which can be either heavy ( >= 250 ms in duration) or light (< 250 ms).
After the page load is complete we analyze these heavy and light task clusters following these steps:
- Find interactive windows where there are no large groups of "densely packed" long tasks. Long tasks must be grouped in a "cluster" < 250 ms long and there must be at least 1 s of padding between these clusters.
- Starting at First Meaningful Paint, find an interactive window that is 5 seconds* long
- First CPU Idle is the start of this first interactive window.
*The required window size shrinks over time. It starts at 5000 ms and shrinks to 3000 ms 15 seconds after First Meaningful Paint.
Why don't I see interactive timing data for my Firefox Real Browser check?
Currently Time to Interactive and First CPU Idle are only available for Chrome checks. You can update your check to use Chrome at any time via the Advanced settings on the check edit page.
Why do my interactive timings look different in Splunk Synthetic Monitoring than in another performance applications?
Our approach is consistent with other performance applications that report interactive timings, but there is no standard definition or method to calculate interactive timings, so implementation details can vary and lead to different results. Time to Interactive and First CPU Idle are also dependent on multiple other metrics (First Contentful and Meaningful Paint, DOM Load, etc), so any difference in those values can amplify differences in interactive timings.
Why don't I see interactive timing data for my Chrome Real Browser check?
Both Time to Interactive and First CPU Idle are dependent on other metrics. Time to Interactive requires First Contentful Paint and DOM Load, and First CPU Idle requires First Meaningful Paint and DOM Load. If any of those metrics are missing (e.g. if the page is blank and no Paint events occur), there will be no value for the corresponding interactive metric. If you see missing or null values for your interactive timings, make sure the required metrics are present.