Consolidate Lighthouse CI report
Motivation
I think the Lighthouse CI report would be more relevant if it was doing multiple runs, showing average scores + variance (if !== 0)
I tried to enable this but unfortunately the reporting does not aggregate multiple runs, so I reverted it in https://github.com/facebook/docusaurus/pull/9448 for now.
We should improve the reporting to consolidate results for each url in a less verbose way, and re-enable "numberOfRuns": 5
It could be convenient to keep reports of each run accessible: for example with small links like 1 2 3 4 5
Also the pathname link now targets localhost. That would be convenient to link to the netlify preview in this case, although it's not in practice the url we used to run lighthouse (maybe add another column to avoid confusion?)
Ideally we'd also want results for both desktop and mobile? (see https://github.com/facebook/docusaurus/issues/9379#issuecomment-1779662595)
Example verbose report
We want to avoid this very long table:
| URL | Performance | Accessibility | Best Practices | SEO | PWA | Report |
|---|---|---|---|---|---|---|
| / | 🟠 66 | 🟢 97 | 🟢 92 | 🟢 100 | 🟠 89 | Report |
| / | 🟠 82 | 🟢 97 | 🟢 92 | 🟢 100 | 🟠 89 | Report |
| / | 🟠 76 | 🟢 97 | 🟢 92 | 🟢 100 | 🟠 89 | Report |
| / | 🟠 80 | 🟢 97 | 🟢 92 | 🟢 100 | 🟠 89 | Report |
| /docs/installation | 🟠 68 | 🟢 98 | 🟢 92 | 🟢 100 | 🟠 89 | Report |
| /docs/installation | 🟠 52 | 🟢 98 | 🟢 92 | 🟢 100 | 🟠 89 | Report |
| /docs/installation | 🟠 52 | 🟢 98 | 🟢 92 | 🟢 100 | 🟠 89 | Report |
| /docs/installation | 🟠 51 | 🟢 98 | 🟢 92 | 🟢 100 | 🟠 89 | Report |
| /docs/category/getting-started | 🟠 64 | 🟢 100 | 🟢 92 | 🟢 90 | 🟠 89 | Report |
| /docs/category/getting-started | 🟠 58 | 🟢 100 | 🟢 92 | 🟢 90 | 🟠 89 | Report |
| /docs/category/getting-started | 🟠 65 | 🟢 100 | 🟢 92 | 🟢 90 | 🟠 89 | Report |
| /docs/category/getting-started | 🟠 63 | 🟢 100 | 🟢 92 | 🟢 90 | 🟠 89 | Report |
| /blog | 🟠 78 | 🟢 100 | 🟢 92 | 🟢 90 | 🟠 89 | Report |
| /blog | 🟠 82 | 🟢 100 | 🟢 92 | 🟢 90 | 🟠 89 | Report |
| /blog | 🟠 57 | 🟢 100 | 🟢 92 | 🟢 90 | 🟠 89 | Report |
| /blog | 🟠 58 | 🟢 100 | 🟢 92 | 🟢 90 | 🟠 89 | Report |
| /blog/preparing-your-site-for-docusaurus-v3 | 🟠 53 | 🟢 97 | 🟢 92 | 🟢 100 | 🟠 89 | Report |
| /blog/preparing-your-site-for-docusaurus-v3 | 🟠 54 | 🟢 97 | 🟢 92 | 🟢 100 | 🟠 89 | Report |
| /blog/preparing-your-site-for-docusaurus-v3 | 🟠 65 | 🟢 97 | 🟢 92 | 🟢 100 | 🟠 89 | Report |
| /blog/preparing-your-site-for-docusaurus-v3 | 🟠 69 | 🟢 97 | 🟢 92 | 🟢 100 | 🟠 89 | Report |
| /blog/tags/release | 🟠 58 | 🟢 100 | 🟢 92 | 🟠 80 | 🟠 89 | Report |
| /blog/tags/release | 🟠 78 | 🟢 100 | 🟢 92 | 🟠 80 | 🟠 89 | Report |
| /blog/tags/release | 🟠 61 | 🟢 100 | 🟢 92 | 🟠 80 | 🟠 89 | Report |
| /blog/tags/release | 🟠 60 | 🟢 100 | 🟢 92 | 🟠 80 | 🟠 89 | Report |
| /blog/tags | 🟠 58 | 🟢 100 | 🟢 92 | 🟢 90 | 🟠 89 | Report |
| /blog/tags | 🟠 64 | 🟢 100 | 🟢 92 | 🟢 90 | 🟠 89 | Report |
| /blog/tags | 🟠 60 | 🟢 100 | 🟢 92 | 🟢 90 | 🟠 89 | Report |
| /blog/tags | 🟠 62 | 🟢 100 | 🟢 92 | 🟢 90 | 🟠 89 | Report |
| / | 🟠 59 | 🟢 97 | 🟢 92 | 🟢 100 | 🟠 89 | Report |
| /docs/installation | 🟠 51 | 🟢 98 | 🟢 92 | 🟢 100 | 🟠 89 | Report |
| /docs/category/getting-started | 🟠 62 | 🟢 100 | 🟢 92 | 🟢 90 | 🟠 89 | Report |
| /blog | 🟠 59 | 🟢 100 | 🟢 92 | 🟢 90 | 🟠 89 | Report |
| /blog/preparing-your-site-for-docusaurus-v3 | 🟠 67 | 🟢 97 | 🟢 92 | 🟢 100 | 🟠 89 | Report |
| /blog/tags/release | 🟠 80 | 🟢 100 | 🟢 92 | 🟠 80 | 🟠 89 | Report |
| /blog/tags | 🟠 61 | 🟢 100 | 🟢 92 | 🟢 90 | 🟠 89 | Report |
Damn, I think this lighthouse on localhost doesn't work at all actually 😅
Due to using pull_request_target we are always running it against the checkout from main branch instead of PR branch
To make this work we'd have to split this workflow in 2:
- build + lighthouse with
pull_request - post comment with
pull_request_target
Or maybe it's safe to keep pull_request and add this permission instead?
permissions:
contents: read
pull-requests: write