GoogleChrome
Add warning that simulated throttling doesn't seem to throttle User Timing measures
- Run LH on Https://www.target.com using Simulated
- Compare to running on Applied
What is the current behavior?
Note that most timings are <2s and faster than even FCP as reported at the top of the report. When run on applied, the numbers are 4+x slower matching what you would expect from the report.
What is the expected behavior?
Timings on the page should line up such that custom timings are also throttled.
This is important for developers that build custom dashboards pulling timings from a LH run. Graphing out all timings from the LH report would lead to an incorrect representation of the order the timings fire (ex: A Custom Hero-timing could fire before FCP)
Environment Information
- Affected Channels: CLI, Extension, Dev Tools
- Lighthouse version: 5.4.0
Related issues
Thanks for filing @cheneytsai!
We haven't considered supporting user timing manipulation in simulated before, but you raise a good point about consistency. There's a lot to think through how to handle them, and it's a tricky problem to solve, so I imagine if custom user timings are important you'll want to be running with applied throttling for the near future.
Example issues we'll need to think through:
- How are start/end marks handled within a task vs. between tasks?
- How is a duration time handled? Do we apply the basic multiplier or try to resimulate the start/end marks?
- What if the start and end are in different tasks? What if the simulated ordering has them reversed? Do we create dependencies between them?
Example of how we're handling this in long tasks diagnostic https://github.com/GoogleChrome/lighthouse/pull/10942
I expect this won't work with user timings though because they are not necessarily tied to a specific task. A user could be marking a network response or passage of wall time. I don't expect this is truly possible to fix in simulated mode as it's impossible to tell if a particular user timing measure should be throttled.
After doing some research here. I don't think we'll ever be able to solve this problem as many user timings have no guarantee of being preserved in a reorganized simulation with different execution. I think the best we can do here is add a warning to the audit if user timings were detected in a simulation that "timings may not match the metric timelines due to simulation" with a link to learn about simulated throttling.
im interested please assign me, ill share my proposed solution as soon as u ping me
Let's just update the docs https://developer.chrome.com/docs/lighthouse/performance/user-timings/