docusaurus icon indicating copy to clipboard operation
docusaurus copied to clipboard

Consolidate Lighthouse CI report

Open slorber opened this issue 2 years ago • 2 comments

Motivation

I think the Lighthouse CI report would be more relevant if it was doing multiple runs, showing average scores + variance (if !== 0)

I tried to enable this but unfortunately the reporting does not aggregate multiple runs, so I reverted it in https://github.com/facebook/docusaurus/pull/9448 for now.

We should improve the reporting to consolidate results for each url in a less verbose way, and re-enable "numberOfRuns": 5

It could be convenient to keep reports of each run accessible: for example with small links like 1 2 3 4 5

Also the pathname link now targets localhost. That would be convenient to link to the netlify preview in this case, although it's not in practice the url we used to run lighthouse (maybe add another column to avoid confusion?)

Ideally we'd also want results for both desktop and mobile? (see https://github.com/facebook/docusaurus/issues/9379#issuecomment-1779662595)

Example verbose report

We want to avoid this very long table:

URL Performance Accessibility Best Practices SEO PWA Report
/ 🟠 66 🟢 97 🟢 92 🟢 100 🟠 89 Report
/ 🟠 82 🟢 97 🟢 92 🟢 100 🟠 89 Report
/ 🟠 76 🟢 97 🟢 92 🟢 100 🟠 89 Report
/ 🟠 80 🟢 97 🟢 92 🟢 100 🟠 89 Report
/docs/installation 🟠 68 🟢 98 🟢 92 🟢 100 🟠 89 Report
/docs/installation 🟠 52 🟢 98 🟢 92 🟢 100 🟠 89 Report
/docs/installation 🟠 52 🟢 98 🟢 92 🟢 100 🟠 89 Report
/docs/installation 🟠 51 🟢 98 🟢 92 🟢 100 🟠 89 Report
/docs/category/getting-started 🟠 64 🟢 100 🟢 92 🟢 90 🟠 89 Report
/docs/category/getting-started 🟠 58 🟢 100 🟢 92 🟢 90 🟠 89 Report
/docs/category/getting-started 🟠 65 🟢 100 🟢 92 🟢 90 🟠 89 Report
/docs/category/getting-started 🟠 63 🟢 100 🟢 92 🟢 90 🟠 89 Report
/blog 🟠 78 🟢 100 🟢 92 🟢 90 🟠 89 Report
/blog 🟠 82 🟢 100 🟢 92 🟢 90 🟠 89 Report
/blog 🟠 57 🟢 100 🟢 92 🟢 90 🟠 89 Report
/blog 🟠 58 🟢 100 🟢 92 🟢 90 🟠 89 Report
/blog/preparing-your-site-for-docusaurus-v3 🟠 53 🟢 97 🟢 92 🟢 100 🟠 89 Report
/blog/preparing-your-site-for-docusaurus-v3 🟠 54 🟢 97 🟢 92 🟢 100 🟠 89 Report
/blog/preparing-your-site-for-docusaurus-v3 🟠 65 🟢 97 🟢 92 🟢 100 🟠 89 Report
/blog/preparing-your-site-for-docusaurus-v3 🟠 69 🟢 97 🟢 92 🟢 100 🟠 89 Report
/blog/tags/release 🟠 58 🟢 100 🟢 92 🟠 80 🟠 89 Report
/blog/tags/release 🟠 78 🟢 100 🟢 92 🟠 80 🟠 89 Report
/blog/tags/release 🟠 61 🟢 100 🟢 92 🟠 80 🟠 89 Report
/blog/tags/release 🟠 60 🟢 100 🟢 92 🟠 80 🟠 89 Report
/blog/tags 🟠 58 🟢 100 🟢 92 🟢 90 🟠 89 Report
/blog/tags 🟠 64 🟢 100 🟢 92 🟢 90 🟠 89 Report
/blog/tags 🟠 60 🟢 100 🟢 92 🟢 90 🟠 89 Report
/blog/tags 🟠 62 🟢 100 🟢 92 🟢 90 🟠 89 Report
/ 🟠 59 🟢 97 🟢 92 🟢 100 🟠 89 Report
/docs/installation 🟠 51 🟢 98 🟢 92 🟢 100 🟠 89 Report
/docs/category/getting-started 🟠 62 🟢 100 🟢 92 🟢 90 🟠 89 Report
/blog 🟠 59 🟢 100 🟢 92 🟢 90 🟠 89 Report
/blog/preparing-your-site-for-docusaurus-v3 🟠 67 🟢 97 🟢 92 🟢 100 🟠 89 Report
/blog/tags/release 🟠 80 🟢 100 🟢 92 🟠 80 🟠 89 Report
/blog/tags 🟠 61 🟢 100 🟢 92 🟢 90 🟠 89 Report

slorber avatar Oct 25 '23 10:10 slorber

Damn, I think this lighthouse on localhost doesn't work at all actually 😅

Due to using pull_request_target we are always running it against the checkout from main branch instead of PR branch

To make this work we'd have to split this workflow in 2:

  • build + lighthouse with pull_request
  • post comment with pull_request_target

Or maybe it's safe to keep pull_request and add this permission instead?

permissions:
  contents: read
  pull-requests: write

slorber avatar Oct 25 '23 14:10 slorber