Rethinking vulnerability check scoring logic: distinguishing between "no vulnerabilities" and "no disclosure"

[gcanlin]

Description

Currently, the Vulnerability check in Scorecard relies on osv-scanner to detect known vulnerabilities. The scoring logic is:

• The fewer unpatched vulnerabilities, the higher the score.
• Projects without detected vulnerabilities receive a high score.

However, this may lead to a misleading result:
Some open-source projects may not have a vulnerability disclosure mechanism or the ability to report/fix vulnerabilities. As a result, such projects will appear to have "no vulnerabilities," thus receiving a high score, even though the absence of reported vulnerabilities may simply mean that vulnerabilities are never disclosed.

Motivation

• Avoid artificially high scores for projects without vulnerability disclosure.
• Encourage communities to establish proper vulnerability reporting and fixing practices.
• Reward projects that actively fix known vulnerabilities.

Proposed Scoring Logic

• Scan all vulnerabilities (including patched ones).
• If no vulnerabilities are found at all → assign -1 score (reflecting lack of evidence, possible missing disclosure).
• If vulnerabilities exist and all are fixed → assign full score 10.
• If vulnerabilities exist and some remain unpatched → deduct score based on the number of unpatched vulnerabilities.

Alternatives considered

• Keep current logic (projects with no vulnerabilities always score high).
• Add an auxiliary check for vulnerability disclosure practices.
• However, directly adjusting the scoring logic as above is simpler and immediately effective.

Willingness to contribute

I am willing to work on this enhancement and submit a PR if the community agrees with this direction.