httparchive.org icon indicating copy to clipboard operation
httparchive.org copied to clipboard

Published 20 hours ago • verified publisher icon HTTPArchive

Replace the Loading Speed report with CrUX

Open rviscomi opened this issue 6 years ago • 4 comments

https://httparchive.org/reports/loading-speed

This report is based on HTTP Archive's synthetic monitoring of the following performance metrics:

  • First Contentful Paint
  • Time to Interactive
  • First CPU Idle
  • DOMContentLoaded
  • onLoad
  • JavaScript Boot-up Time

It's not our intent to draw conclusions about real users' experiences based on synthetic metrics like these. In order to avoid this pitfall, we should replace the metrics in the report with the RUM-based metrics from CrUX.

To implement this, we should overwrite the existing loading-speed report with the metrics in the chrome-ux-report.

rviscomi avatar Jun 18 '19 03:06 rviscomi

Strong +1. It's far too easy to draw wrong conclusions based on current synthetic tests running on oversubscribed VMs.

igrigorik avatar Jun 18 '19 05:06 igrigorik

Strong +1. It's far too easy to draw wrong conclusions based on current synthetic tests running on oversubscribed VMs.

FWIW, WPT uses n1-standard-2, which should be fine according to the Lighthouse recommended machine specs. It'd be concerning if it were using n1-standard-1. (And of course a beefier machine may improve variability, but we currently haven't collected data to quantify the degree.)

paulirish avatar Mar 31 '20 21:03 paulirish