checkly
Set throttling of test
Is your feature request related to a problem? Please describe.
Core Web Vitals ( with Lighthouse 9.3.0) uses an Emulated Moto G4 with Slow 4G throttling as a base line. With your current implementation a page can have very good metrics because there is no throttling in place. On a test I am doing TBT is 400ms on your platform and 4500ms on lighthouse.
Also, your platform doesn't show the Time to interactivity which is another important metric, btw your note in the dashboard is a bit misleading: "your initial page load becomes responsive to user input" but the label is TBT: Total Blocking Time
.
Describe the solution you'd like
- being able to configure multiple throttling per test
- Display Time to Interactive and speed index in your dashboard.
Describe alternatives you've considered Another product ? or can we use this: https://github.com/microsoft/playwright/issues/6038#issuecomment-812521882
Hi @cjtrbx, thanks for contributing. The Lighthouse suite is a bit different than the basic web vitals we give you for the following reasons:
-
Lighthouse is very resource intensive and not recommended to run on serverless or containerized resources. See https://github.com/GoogleChrome/lighthouse/blob/master/docs/variability.md
-
Running Lighthouse tests is therefore not cost-effective within our current pricing model. We could run it, but the results would not be reliable enough. Does that mean we will never have full Lighthouse checks? No, we would love to have them but we have no ETA yet on that.
Having said that, you can totally set a throttling configuration. The playwright issue you linked should work. Also see https://github.com/microsoft/playwright/issues/8622
On TBT vs. TTI. Measuring TTI in a lab situation is not recommended. It requires interaction, which we cannot guarantee. For this reason, we measure TBT which we can actually measure without an interaction from a user. As per the tooltip, the "TBT measures the amount of time between First Contentful Paint (FCP) and Time to Interactive (TTI) where the main thread was blocked and prevented input responsiveness."
