sitespeedio
Enhance `Navigation timing` metrics. Split absolute and relative metrics, and add new metrics into the table
Feature/improvement
Version
sitespeedio/sitespeed.io:37.5.1-plus1
Current state
Navigation metrics in a report
We have a table with navigation metrics
Navigation Timing
| Metric | Value |
|---|---|
| backEndTime | 186 ms |
| domContentLoadedTime | 489 ms |
| domInteractiveTime | 489 ms |
| domainLookupTime | 2 ms |
| frontEndTime | 304 ms |
| pageDownloadTime | 0 ms |
| pageLoadTime | 490 ms |
| redirectionTime | 0 ms |
| serverConnectionTime | 76 ms |
| serverResponseTime | 104 ms |
Navigation metrics in browsers
https://developer.mozilla.org/en-US/docs/Web/API/PerformanceNavigationTiming
https://mdn.github.io/shared-assets/images/diagrams/api/performance/timestamp-diagram.svg
Improvments
Split absolute and relative metrics
There are absolute metrics from the report. It is a diff between the event and 0 (startTime == 0 always)
| Metric | Value |
|---|---|
| backEndTime | 186 ms |
| domContentLoadedTime | 489 ms |
| domInteractiveTime | 489 ms |
| ~~domainLookupTime~~ | 2 ms |
| ~~frontEndTime~~ | 304 ms |
| ~~pageDownloadTime~~ | 0 ms |
| pageLoadTime | 490 ms |
| redirectionTime | 0 ms |
| ~~serverConnectionTime~~ | 76 ms |
| ~~serverResponseTime~~ | 104 ms |
The metric pageLoadTime is important here, because it is a full time from start to end. But all other metrics are not important, because they are absolute.
For example the backEndTime is
The time it takes for the network and the server to generate and start sending the HTML. Collected using the Navigation Timing API with the definition: responseStart - navigationStart
It is not about a duration of a backend side, it is about a staring point of a request to the backend side. It is not clear
A table with a title Navigation moments or Navigation starting moments will be better
| Metric | Value |
|---|---|
| backEndTime | 186 ms |
| domContentLoadedTime | 489 ms |
| domInteractiveTime | 489 ms |
| pageLoadTime | 490 ms |
| redirectionTime | 0 ms |
There are relative metrics from the report. It is a diff between the event and some previous event:
| Metric | Value |
|---|---|
| ~~backEndTime~~ | 186 ms |
| ~~domContentLoadedTime~~ | 489 ms |
| ~~domInteractiveTime~~ | 489 ms |
| domainLookupTime | 2 ms |
| frontEndTime | 304 ms |
| pageDownloadTime | 0 ms |
| ~~pageLoadTime~~ | 490 ms |
| ~~redirectionTime~~ | 0 ms |
| serverConnectionTime | 76 ms |
| serverResponseTime | 104 ms |
Will be better to separate them, and add a clear title like Navigation durations or Navigation steps or Navigation intervals
| Metric | Value |
|---|---|
| domainLookupTime | 2 ms |
| frontEndTime | 304 ms |
| pageDownloadTime | 0 ms |
| serverConnectionTime | 76 ms |
| serverResponseTime | 104 ms |
The second suggestion is about adding new steps/intervals about the Navigation timing. I'm talking about blocks and connections between the blocks of the diagram:
- https://mdn.github.io/shared-assets/images/diagrams/api/performance/timestamp-diagram.svg
I used a custom script defaultPageCompleteCheck.js for getting all navigation steps from the specification:
defaultPageCompleteCheck.js
return (function(waitTime) {
try {
console.log(`defaultPageCompleteCheck: waitTime ${waitTime}`);
var end = window.performance.timing.loadEventEnd;
console.log(`defaultPageCompleteCheck: end ${end}`);
var start= window.performance.timing.navigationStart;
console.log(`defaultPageCompleteCheck: start ${start}`);
console.log(`defaultPageCompleteCheck: end > 0 : ${end > 0}`);
console.log(`defaultPageCompleteCheck: end - start + waitTime : ${end - start + waitTime}`);
console.log(`defaultPageCompleteCheck: performance.now() : ${performance.now()}`);
console.log(`defaultPageCompleteCheck: return 1 : ${ (end > 0) && (performance.now() > end - start + waitTime) }`);
let status = (end > 0) && (performance.now() > end - start + waitTime);
if(status) {
try {
const entries = performance.getEntriesByType("navigation");
const entry = entries[0];
performance.measure( "0_start_end", {
start: 0,
duration: window.performance.timing.loadEventEnd - window.performance.timing.navigationStart,
});
performance.measure( "0_startTime_loadEventEnd", {
start: 0,
duration: entry.loadEventEnd - entry.startTime,
});
performance.mark( "0_start_end", {
startTime: window.performance.timing.loadEventEnd - window.performance.timing.navigationStart,
});
performance.mark( "0_startTime_loadEventEnd", {
startTime: entry.loadEventEnd,
});
let index = 1;
performance.measure(`${index++}_startTime_redirectStart`,
{
start: entry.startTime,
duration: entry.redirectStart - entry.startTime,
}
);
performance.measure(`${index++}_redirectStart_redirectEnd`,
{
start: entry.redirectStart,
duration: entry.redirectEnd - entry.redirectStart,
}
);
performance.measure(`${index++}_redirectEnd_workerStart`,
{
start: entry.redirectEnd,
duration: entry.workerStart - entry.redirectEnd,
}
);
performance.measure(`${index++}_workerStart_fetchStart`,
{
start: entry.workerStart,
duration: entry.fetchStart - entry.workerStart,
}
);
performance.measure(`${index++}_fetchStart_domainLookupStart`,
{
start: entry.fetchStart,
duration: entry.domainLookupStart - entry.fetchStart,
}
);
performance.measure(`${index++}_domainLookupStart_domainLookupEnd`,
{
start: entry.domainLookupStart,
duration: entry.domainLookupEnd - entry.domainLookupStart,
}
);
performance.measure(`${index++}_domainLookupEnd_connectStart`,
{
start: entry.domainLookupEnd,
duration: entry.connectStart - entry.domainLookupEnd,
}
);
performance.measure(`${index++}_connectStart_secureConnectionStart`,
{
start: entry.connectStart,
duration: entry.secureConnectionStart - entry.connectStart,
}
);
performance.measure(`${index++}_secureConnectionStart_connectEnd`,
{
start: entry.secureConnectionStart,
duration: entry.connectEnd - entry.secureConnectionStart,
}
);
performance.measure(`${index++}_connectStart_connectEnd`,
{
start: entry.connectStart,
duration: entry.connectEnd - entry.connectStart,
}
);
performance.measure(`${index++}_connectEnd_requestStart`,
{
start: entry.connectEnd,
duration: entry.requestStart - entry.connectEnd,
}
);
performance.measure(`${index++}_requestStart_responseStart`,
{
start: entry.requestStart,
duration: entry.responseStart - entry.requestStart,
}
);
performance.measure(`${index++}_responseStart_finalResponseHeadersStart`,
{
start: entry.requestStart,
duration: entry.finalResponseHeadersStart - entry.responseStart,
}
);
performance.measure(`${index++}_finalResponseHeadersStart_responseEnd`,
{
start: entry.finalResponseHeadersStart,
duration: entry.responseEnd - entry.finalResponseHeadersStart,
}
);
performance.measure(`${index++}_finalResponseHeadersStart_responseEnd`,
{
start: entry.finalResponseHeadersStart,
duration: entry.responseEnd - entry.finalResponseHeadersStart,
}
);
performance.measure(`${index++}_responseStart_responseEnd`,
{
start: entry.responseStart,
duration: entry.responseEnd - entry.responseStart,
}
);
performance.measure(`${index++}_unloadEventStart_unloadEventEnd`,
{
start: entry.unloadEventStart,
duration: entry.unloadEventEnd - entry.unloadEventStart,
}
);
performance.measure(`${index++}_responseEnd_domInteractive`,
{
start: entry.responseEnd,
duration: entry.domInteractive - entry.responseEnd,
}
);
performance.measure(`${index++}_domInteractive_domContentLoadedEventStart`,
{
start: entry.domInteractive,
duration: entry.domContentLoadedEventStart - entry.domInteractive,
}
);
performance.measure(`${index++}_domContentLoadedEventStart_domContentLoadedEventEnd`,
{
start: entry.domContentLoadedEventStart,
duration: entry.domContentLoadedEventEnd - entry.domContentLoadedEventStart,
}
);
performance.measure(`${index++}_domContentLoadedEventEnd_domComplete`,
{
start: entry.domContentLoadedEventEnd,
duration: entry.domComplete - entry.domContentLoadedEventEnd,
}
);
performance.measure(`${index++}_domComplete_loadEventStart`,
{
start: entry.domComplete,
duration: entry.loadEventStart - entry.domComplete,
}
);
performance.measure(`${index++}_loadEventStart_loadEventEnd`,
{
start: entry.loadEventStart,
duration: entry.loadEventEnd - entry.loadEventStart,
}
);
const measures_my = [];
const marks_my = [];
if (window.performance && window.performance.getEntriesByType) {
const myMarks = Array.prototype.slice.call(
window.performance.getEntriesByType('mark')
);
for (const mark of myMarks) {
if (mark.detail && mark.detail.devtools) {
continue;
} else {
marks_my.push({
name: mark.name,
startTime: mark.startTime
});
}
}
const myMeasures = Array.prototype.slice.call(
window.performance.getEntriesByType('measure')
);
for (const measure of myMeasures) {
if (measure.detail && measure.detail.devtools) {
continue;
} else {
measures_my.push({
name: measure.name,
duration: measure.duration,
startTime: measure.startTime
});
}
}
}
console.log(`defaultPageCompleteCheck: measures_my: \n${JSON.stringify(measures_my)}`)
console.log(`defaultPageCompleteCheck: marks_my: \n${JSON.stringify(marks_my)}`)
} catch (e) {
console.log(`defaultPageCompleteCheck: exception: \n${JSON.stringify(e)}`)
}
return true;
} else {
return false;
}
}
catch(e) {
console.log(`defaultPageCompleteCheck: e : ${JSON.stringify(e)}`);
console.log(`defaultPageCompleteCheck: return 2 : true`);
return true;
}
})(arguments[arguments.length - 1]);
There are my outcomes:
The longest steps/intervals for Chrome pages with an empty cache are:
- responseEnd ... domInteractive or
18_responseEnd_domInteractiveup to 70% - requestStart ... responseStart or
12_requestStart_responseStartup to 10-20% - connectStart ... secureConnectionStart or
8_connectStart_secureConnectionStart10-20% - secureConnectionStart ... connectEnd or
9_secureConnectionStart_connectEnd10-20% - domContentLoadedEventEnd ... domComplete or
21_domContentLoadedEventEnd_domCompleteit is equal to 0 for 99% of all tests, but it is 50% for 1% of tests
I think we can add those TOP-5 navigation interval as well, or we can add all intervals into the report
By the way, what do you think about the intervals:
- responseEnd ... domInteractive or
18_responseEnd_domInteractiveup to 70% - domContentLoadedEventEnd ... domComplete or
21_domContentLoadedEventEnd_domCompleteit is equal to 0 for 99% of all tests, but it is 50% for 1% of tests
@soulgalore
?
There are two invisible intervals from the specification, but there are the longest intervals for the Chromium 136.0.7103.25 metrics. How is it possible?
Is it an issue of the Chromium 136.0.7103.25 ?
Because the interval responseEnd ... domInteractive has significant improvements for the repeat-tests (the tests with a hot cache):
The interval domContentLoadedEventStart ... domContentLoadedEventEnd or 20_domContentLoadedEventStart_domContentLoadedEventEnd should have improvements for the repeat tests, but this interval is equal to 0 for all tests.
I think the Chromium 136.0.7103.25 has an issue, it collects the DOM-Tree-duration as responseEnd ... domInteractive instead of domContentLoadedEventStart ... domContentLoadedEventEnd
My motivation for adding intervals / steps / blocks
We can visualize specific blocks via the Bar Chart diagram in Grafana, for the example the TCP block:
Sorry I don't follow along, what do you want to do? :) Can you explain what you want to do and what you will gain with it? We collect the Navigation Timing raw data in https://github.com/sitespeedio/browsertime/blob/main/browserscripts/timings/navigationTiming.js and it's sent to Graphite by default, so I think you can create the bar charts using those if they are useful for you.
The metrics in https://github.com/sitespeedio/browsertime/blob/main/browserscripts/timings/pageTimings.js is how GA reported metrics long time ago. I kept those in the HTML report and then use the raw metrics only in the time series database so you can create graphs for it. Maybe we could include those raw values too if we can make it clean in the HTML.
There are two invisible intervals from the specification
I would check that with the Chrome team if its not in the specification.
