Report GitHub check status differently when highest failing assert is a warning

[bcmedeiros]

I've currently set a assert profile like this:

module.exports = {
  ci: {
    ...
    assert: {
      ...
      assertions: {
        "categories:performance": "warn",
        "categories:best-practices": "warn",
        "categories:seo": "warn",
        "uses-long-cache-ttl": "off",
        "uses-http2": "off"
      }
    }
  }
}

What I get on github is this:

The message is fine, but having a success conclusion makes the checks panel to auto-collapse, so users won't be attracted to it at all. Maybe we should use neutral or find another way to improve that, but without failing the check?

[bcmedeiros]

If neutral is not a good idea, maybe we should at least keep the original text when there are no asserts, something like:

2 warning(s) - Performance: 66, Accessibility: 94, Best Practices: 92, SEO: 79, Progressive Web App: 64

[patrickhulce]

Thanks for filing @brunojcm?

I wasn't aware neutral is an option. How does it appear in the GitHub UI? I'm not sure how to set it or whether it would satisfy the "required" status checks the same way :/ https://developer.github.com/v3/repos/statuses/#create-a-commit-status

maybe we should at least keep the original text when there are no asserts, something like:

I'm fine with this 👍 PR welcome :)

[ognjenjevremovic]

Hey @patrickhulce 👋

I wanted to contribute to the project and saw this issue open and labeled as a good first issue. Since I didn't make any PR previously to lighthouse-ci (nor any project from the lighthouse suite of packages), I think this issue might be a good start for the first time contributors to the project (such as me). Can you confirm that issue is still open (with no work being done) and is up for grabs? 🙂

Also, it would be great if you could please point me in the right direction and perhaps provide more info on where one could start in order to successfully implement this 🙂 .

Many thanks! 🎉

[patrickhulce]

It's still open @ognjenjevremovic thanks for wanting to help!

You'll want to take a look at the following block of code where this happens.

https://github.com/GoogleChrome/lighthouse-ci/blob/7f46602f1e2d86effcda994ec951664982e3dc8a/packages/cli/src/upload/upload.js#L254-L291

The task here is to extract the logic that generates the string with Performance: <XX>, Accessibility: <XX> and use it in both branches of that if (with and without assertions). A unit test asserting as much would be great to add as well :)