WebKit
Speedometer doesn't capture all React work
The react scheduler does work until a deadline is reached, then uses postMessage to schedule additional work. The speedometer runner does not wait for this additional work to complete before finishing the async period of the test. This is a general problem, but differences in timing between Safari and Chrome mean that Chrome ends up doing more real work during the scoring phase of the test.
To showcase the challenge, let's look at the NewsSite-Next workload, where we added a debug message in a component:
useEffect(() => {
console.log(`useEffect with message: ${title}`);
performance.mark(`useEffect with message: ${title}`);
});
useLayoutEffect(() => {
console.log(`useLayoutEffect with message: ${title}`);
performance.mark(`useLayoutEffect with message: ${title}`);
});
Screenshot from Chrome:
Screenshot from Safari:
A potential solution could be to use an AsyncSuiteRunner class that waits for each test.run() step. I tested the approach locally and do see a more fair comparison between Chrome and Safari, with the message being captured in the test steps during the measured async time.
Safari before:
Safari after:
Speedometer3 NewsSite-Next correctness fix is a document describing the score correctness issue in more detail, and https://github.com/WebKit/Speedometer/pull/458 is the pull request that fixes this.
@rniwa, just FYI that this PR fixes this issue. Sorry for not making this clearer in the original issue text.
