web-vitals

web-vitals copied to clipboard

Looks in Firefox Nightly FCP is often quite slow in our e2e tests.

For example this run has this when I turn on debugging:

[0-1] reports the correct value after the first paint {
[0-1]   name: 'FCP',
[0-1]   value: 53[15](https://github.com/GoogleChrome/web-vitals/actions/runs/8923549678/job/24508034241#step:6:16),
[0-1]   rating: 'poor',
[0-1]   delta: 5315,
[0-1]   entries: [
[0-1]     {
[0-1]       name: 'first-contentful-paint',
[0-1]       entryType: 'paint',
[0-1]       startTime: 5315,
[0-1]       duration: 0
[0-1]     }
[0-1]   ],
[0-1]   id: 'v4-[17](https://github.com/GoogleChrome/web-vitals/actions/runs/8923549678/job/24508034241#step:6:18)14651853858-9340183650873',
[0-1]   navigationType: 'navigate'
[0-1] }

A rerun often solves it as you can see later in that log:

[0-1] reports the correct value after the first paint { [0-1] name: 'FCP', [0-1] value: 72, [0-1] rating: 'good', [0-1] delta: 72, [0-1] entries: [ [0-1] { [0-1] name: 'first-contentful-paint', [0-1] entryType: 'paint', [0-1] startTime: 72, [0-1] duration: 0 [0-1] } [0-1] ], [0-1] id: 'v4-1714651859617-3386553824299', [0-1] navigationType: 'navigate' [0-1] }

But it doesn't seem to solve it for these tests which have been continually failing:
- FCP - 'reports if the page is restored from bfcache'
- LCP - 'reports the correct value on hidden (reportAllChanges === false)'

I hoped it was just some nightly flakiness but doesn't seem to be getting resolved so we might want to comment these out for now for Firefox.

I can't reproduce this on MacOS, nor on linux (Chromebox) so it only seems to happen in GitHub Actions. Possibly related to there not being a display there?

Raised a bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1894695

So #476 resolved a lot of the cases for now but this one is still susceptible to failing intermittently due to the Firefox slowness in GitHub Actions:

https://github.com/GoogleChrome/web-vitals/blob/fd8b0c3a59089c080fdb3ed9297a9e73867f4963/test/e2e/onLCP-test.js#L715-L718

A rerun often fixes it but might take a few attempts. If it gets really annoying we can consider skipping this last line out for Firefox too.

Either way leaving this open to remind us to remove those workarounds once the real fix is released.

Workarounds were removed in #507 so this can be closed