"Vulnerabilities" check: unclear description ("open vulnerabilities")

[Chealer]

The Vulnerabilities check has the following description:

This check determines whether the project has open, unfixed vulnerabilities in its own codebase or its dependencies using the OSV (Open Source Vulnerabilities) service.

It would greatly clarify to define what this calls an "open vulnerability".

[github-actions[bot]]

This issue has been marked stale because it has been open for 60 days with no activity.

[klbynum]

Hey @Chealer, what do you think of our PR #4271 for this issue?

[Chealer]

Hi @klbynum The concern was not identifying vulnerabilities, but to explain what the quoted sentence refers to as "open vulnerabilities". In other words, the description should explain which sense the term "open" has when it qualifies vulnerabilities.

To make an analogy, if there are 5 doors in my home, I am not asking to list that doors #2 and #4 are the ones which are open. What this asks is to define which characteristic(s) these open doors share.

[klbynum]

Thank you for this clarification @Chealer. From what we found, the "open" in "open vulnerabilities" refers to "open source" as the scanner used for this check detects "vulnerabilities in open source packages" as found here. Do you agree with this? If so, would clarifying this in the documentation resolve the issue?

[spencerschrock]

Thank you for this clarification @Chealer. From what we found, the "open" in "open vulnerabilities" refers to "open source" as the scanner used for this check detects "vulnerabilities in open source packages" as found here. Do you agree with this? If so, would clarifying this in the documentation resolve the issue?

I think that is the wrong definition for this context. My thoughts are that the "open" is being used as a synonym to the "unfixed" portion that follows it. It was added in #628, and the summary added to the README simply says:

Does the project have unfixed vulnerabilities?

It may be easier to just remove the word open from the checks.yaml to match that phrasing

[Chealer]

@klbynum: Scorecard is dedicated to open source packages. Given that all checks only find issues in open source software, it is hard to believe that the author would have wasted energy in specifying that and phrased it in such a poor way.

Thank you very much for your research @spencerschrock

@oliverchang : can you explain what you meant by "open vulnerabilities"?

[oliverchang]

Yes, the intent of "open vulnerability" in this context was "unfixed vulnerability".

[Chealer]

Thanks @oliverchang

In that case, I recommend determining how exactly the tool works to clarify what "unfixed" means and to fix the text accordingly, in a way which reflects that OSV only checks for known vulnerabilities. Depending on the result, this could lead to something like:

This check determines whether the project's latest stable version is publically known to be vulnerable―either due to flaws directly in its own codebase, or via dependencies on other vulnerable projects whose latest stable version is vulnerable―using the OSV (Open Source Vulnerabilities) database.

[github-actions[bot]]

This issue has been marked stale because it has been open for 60 days with no activity.