M3-5714: Generate Jest coverage report via GitHub Actions

[DevDW]

Description

Generate a Jest coverage report when running the CI workflow via GitHub Actions and save it as an artifact.

[bnussman]

Really neat! I noticed that it seemed to add 15 min to the test-manager stage of CI which is a bit unfortunate, but not a big deal for now imo

[DevDW]

Really neat! I noticed that it seemed to add 15 min to the test-manager stage of CI which is a bit unfortunate, but not a big deal for now imo

I'll look into seeing if we can shorten that time, but one big issue with the initial attempt was the size of the artifact (70+ MB) because it took all the contents of packages/manager and spit it out in the artifact. Going to see if the previous commit makes any difference on that front, but once that's sorted I'll circle back to the time considerations

[DevDW]

Looks like this adds anywhere from 6-9 minutes to how long the checks take to complete. The artifact also comes in at just under 8 MB.

[jdamore-linode]

I think this is a valid tradeoff -- there aren't a lot of cases where we're anxiously waiting for GHA to finish, and if it does end up becoming a burden we can reevaluate.

[DevDW]

Nice job! I see the coverage artifact! Is there an easy way to consume and view the json? Did we explore outputting html?

Thanks! I could be wrong, but I don't think we can get HTML output easily -- I don't see a flag for it in https://jestjs.io/docs/cli. I tried it manually just in case it was valid but undocumented and the command (yarn workspace linode-manager run test --coverage --outputFile=coverage_report.html --html) failed

[jdamore-linode]

Nice job! I see the coverage artifact! Is there an easy way to consume and view the json? Did we explore outputting html?

Thanks! I could be wrong, but I don't think we can get HTML output easily -- I don't see a flag for it in https://jestjs.io/docs/cli. I tried it manually just in case it was valid but undocumented and the command (yarn workspace linode-manager run test --coverage --outputFile=coverage_report.html --html) failed

There is an HTML report which is pretty nice, but it includes hundreds (thousands?) of small files and stacks up to around 30MB. It gets generated at packages/manager/coverage/lcov-report if you run yarn test --coverage, but I don't know how to generate only the HTML report.

[jdamore-linode]

Perhaps we can export it as JUnit XML and use something like JUnit Report Action?

Edit: This may be more applicable: https://github.com/marketplace/actions/code-coverage-report

[DevDW]

I was initially thinking the "Resource not accessible by integration" error was related to the lcov.info file being in a directory ignored by git, but the error looks to be permissions-related at the GHA integration level: https://www.google.com/search?q=%22Resource+not+accessible+by+integration%22+jest+site%3Agithub.com

Going to continue investigating but this may present a problem:

Pull requests from public forks are still considered a special case and will receive a read token regardless of these settings. (from https://github.blog/changelog/2021-04-20-github-actions-control-permissions-for-github_token/)

update: added Do Not Merge label pending a meeting later this week to investigate implications of some changes we are considering.

[dwiley-akamai]

Per discussions in the team channel in Slack, in light of us pursuing the coverage report via an alternative option, we will not be implementing the GHA approach at this time.