DependencyTrack
Add support for showing OSSF Scorecard scores
Current Behavior
Focus currently is on already known vulnerabilities.
Proposed Behavior
There are projects like https://github.com/ossf/scorecard which calculate a security health score for repositories. Having this score as additional data available would allow several benefits:
- Get aware of low trust components in the project when showing the score in the components table
- Define policies that trigger when using low trust components (below a defineable score), allowing for actions like either exchange the dependency or define further action to be taken (as required by some industry standards)
OSSF Scorecard provides a REST API for the most relevant github projects, but also a public dataset that can be queried (maybe imported?). It should be a feature that can be enabled optionally, to reduce load if one does not need it.
There is another request to integrate deps.dev (https://github.com/DependencyTrack/dependency-track/issues/1499) which also would include OSSF Scorecard values, but this is halted due to missing PURL support in deps.dev. So maybe integrating it separately would make sense for now.
Checklist
- [X] I have read and understand the contributing guidelines
- [X] I have checked the existing issues for whether this enhancement was already requested
Hi,
OSV.dev is asking future additions to https://github.com/google/osv.dev?tab=readme-ov-file#third-party-tools-and-integrations to consider adopting OpenSSF Scorecard and as a part of that, we're also making the request of legacy entrants.
We feel it helps boost the security credibility of the projects and products we're linking to.
Here's the results of a one-time run:
RESULTS
-------
Aggregate score: 6.6 / 10
Check scores:
|---------|------------------------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| SCORE | NAME | REASON | DOCUMENTATION/REMEDIATION |
|---------|------------------------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 10 / 10 | Binary-Artifacts | no binaries found in the repo | https://github.com/ossf/scorecard/blob/3155309aa81adf3395f4d62ee133b524ff316da1/docs/checks.md#binary-artifacts |
|---------|------------------------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 2 / 10 | Branch-Protection | branch protection is not | https://github.com/ossf/scorecard/blob/3155309aa81adf3395f4d62ee133b524ff316da1/docs/checks.md#branch-protection |
| | | maximal on development and all | |
| | | release branches | |
|---------|------------------------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 10 / 10 | CI-Tests | 13 out of 13 merged PRs | https://github.com/ossf/scorecard/blob/3155309aa81adf3395f4d62ee133b524ff316da1/docs/checks.md#ci-tests |
| | | checked by a CI test -- score | |
| | | normalized to 10 | |
|---------|------------------------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 2 / 10 | CII-Best-Practices | badge detected: InProgress | https://github.com/ossf/scorecard/blob/3155309aa81adf3395f4d62ee133b524ff316da1/docs/checks.md#cii-best-practices |
|---------|------------------------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 1 / 10 | Code-Review | Found 1/7 approved changesets | https://github.com/ossf/scorecard/blob/3155309aa81adf3395f4d62ee133b524ff316da1/docs/checks.md#code-review |
| | | -- score normalized to 1 | |
|---------|------------------------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 10 / 10 | Contributors | project has 21 contributing | https://github.com/ossf/scorecard/blob/3155309aa81adf3395f4d62ee133b524ff316da1/docs/checks.md#contributors |
| | | companies or organizations | |
|---------|------------------------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 10 / 10 | Dangerous-Workflow | no dangerous workflow patterns | https://github.com/ossf/scorecard/blob/3155309aa81adf3395f4d62ee133b524ff316da1/docs/checks.md#dangerous-workflow |
| | | detected | |
|---------|------------------------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 10 / 10 | Dependency-Update-Tool | update tool detected | https://github.com/ossf/scorecard/blob/3155309aa81adf3395f4d62ee133b524ff316da1/docs/checks.md#dependency-update-tool |
|---------|------------------------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 0 / 10 | Fuzzing | project is not fuzzed | https://github.com/ossf/scorecard/blob/3155309aa81adf3395f4d62ee133b524ff316da1/docs/checks.md#fuzzing |
|---------|------------------------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 10 / 10 | License | license file detected | https://github.com/ossf/scorecard/blob/3155309aa81adf3395f4d62ee133b524ff316da1/docs/checks.md#license |
|---------|------------------------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 10 / 10 | Maintained | 30 commit(s) and 16 issue | https://github.com/ossf/scorecard/blob/3155309aa81adf3395f4d62ee133b524ff316da1/docs/checks.md#maintained |
| | | activity found in the last 90 | |
| | | days -- score normalized to 10 | |
|---------|------------------------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| ? | Packaging | packaging workflow not | https://github.com/ossf/scorecard/blob/3155309aa81adf3395f4d62ee133b524ff316da1/docs/checks.md#packaging |
| | | detected | |
|---------|------------------------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 8 / 10 | Pinned-Dependencies | dependency not pinned by hash | https://github.com/ossf/scorecard/blob/3155309aa81adf3395f4d62ee133b524ff316da1/docs/checks.md#pinned-dependencies |
| | | detected -- score normalized | |
| | | to 8 | |
|---------|------------------------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 0 / 10 | SAST | SAST tool is not run on all | https://github.com/ossf/scorecard/blob/3155309aa81adf3395f4d62ee133b524ff316da1/docs/checks.md#sast |
| | | commits -- score normalized to | |
| | | 0 | |
|---------|------------------------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 9 / 10 | Security-Policy | security policy file detected | https://github.com/ossf/scorecard/blob/3155309aa81adf3395f4d62ee133b524ff316da1/docs/checks.md#security-policy |
|---------|------------------------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 0 / 10 | Signed-Releases | Project has not signed or | https://github.com/ossf/scorecard/blob/3155309aa81adf3395f4d62ee133b524ff316da1/docs/checks.md#signed-releases |
| | | included provenance with any | |
| | | releases. | |
|---------|------------------------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 10 / 10 | Token-Permissions | GitHub workflow tokens follow | https://github.com/ossf/scorecard/blob/3155309aa81adf3395f4d62ee133b524ff316da1/docs/checks.md#token-permissions |
| | | principle of least privilege | |
|---------|------------------------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| 10 / 10 | Vulnerabilities | 0 existing vulnerabilities | https://github.com/ossf/scorecard/blob/3155309aa81adf3395f4d62ee133b524ff316da1/docs/checks.md#vulnerabilities |
| | | detected | |
|---------|------------------------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|
There are projects like https://github.com/ossf/scorecard which calculate a security health score for repositories.
These scores are repo-based. Even if a repo was known for a specific package, how would a repo translate to a specific version of a package? (PS: especially since a package/version combination is not necessarily bound to a VCS tag/hash...)
(I am asking because I wonder what the actual user story would be, and how a DependencyTrack UI/UX behavior could look/feel like.)
Hmmm, that's a very good point. There were some discussions around supporting multiple sources of health metrics.
Already a complex undertaking, but it will be more complex if those sources are scoped differently (i.e. repo vs. package vs. project, etc.).
These scores are repo-based. Even if a repo was known for a specific package, how would a repo translate to a specific version of a package? (PS: especially since a package/version combination is not necessarily bound to a VCS tag/hash...)
They don't. They would only be on calculated on package level. However, they are a good indicator how trustful your packages in the project are. From UI side I wouldn't mind see some of this data within the component table anyway. Or possibly another tab in the project focusing in health metrics. Doesn't matter if its version specific or not for that indication and for policy configuration. On the pro side, it makes caching easier, as you only need to cache values based on repo addresses, multiple versions can reference the same dataset.
(E.g. Having policies flagging specific packages for review if below thresholds would be great)
However, they are a good indicator how trustful your packages in the project are.
this is debatable. even the OSSF/ScoreCard community agreed that this is not true. But that is just not the point here. If they have any value for you then this is fine. :+1:
My question is: could you shape a user story? What data would you expect to see, and what is the reason behind seeing it, what is the expected use case and value of that visible data?
I'd imagine something like
As a software developer I check the dependencies of my project in dependency track. For each of the project's dependencies (versioned package), I see an overall ScoreCard score, and i can click on that score to ses the details (sub-scores/weight, and a hyperlink(e.g.
https://scorecard.dev/viewer/?uri=github.com/DependencyTrack/frontend) to the data source) of that score for this package, so that I can see the detals and click that link.
the problem with this particular example of a bad user story is: it does not show the value of having this information displayed in the first place. What is the value of "so that i can see ..."?! The reason is that there is no explanation in my example user story describing the use case for them. Here is just one resource on writing good user stories: https://cleancommit.io/blog/how-to-write-a-good-user-story-with-examples-templates/ (Nope, it would be still a bad user story, if you just copied my example and added a proper "so that ...". :-) you should come up with an own one.)
Doesn't matter if its version specific or not for that indication and for policy configuration.
Why? Could you elaborate on such out-of-scope criteria?
My concern: an overall undifferentiated score that is based on "today's" status of a repository - how can this tell me anything about a package's version that was built from some source 2 years ago?
PS: I am not trying to steal the topic, I am just interested in the actual applied value and real-world use cases of OSSF ScoreCard in general and in DependencyTrack in particular.
My apologies, I think in my haste, I misunderstood this issue.
The intent with https://github.com/DependencyTrack/dependency-track/issues/3048#issuecomment-2216553324 was to ask this repo to adopt OpenSSF Scorecard, but I believe this issue is about having the tool report on it.
I will file a new issue.
My question is: could you shape a user story? What data would you expect to see, and what is the reason behind seeing it, what is the expected use case and value of that visible data?
i can think of the following user story:
As a software developer, I want to see project dependencies with OpenSSF score below a particular threshold so that i can avoid them and/or try to replace them by higher scored alternatives and thus reduce the risk of relying on "insufficiently" maintained components or components produced by "insufficient" quality standards
However, they are a good indicator how trustful your packages in the project are.
this is debatable. even the OSSF/ScoreCard community agreed that this is not true. But that is just not the point here. If they have any value for you then this is fine. 👍
My question is: could you shape a user story? What data would you expect to see, and what is the reason behind seeing it, what is the expected use case and value of that visible data?
I'd imagine something like
As a software developer I check the dependencies of my project in dependency track. For each of the project's dependencies (versioned package), I see an overall ScoreCard score, and i can click on that score to ses the details (sub-scores/weight, and a hyperlink(e.g.
https://scorecard.dev/viewer/?uri=github.com/DependencyTrack/frontend) to the data source) of that score for this package, so that I can see the detals and click that link.the problem with this particular example of a bad user story is: it does not show the value of having this information displayed in the first place. What is the value of "so that i can see ..."?! The reason is that there is no explanation in my example user story describing the use case for them. Here is just one resource on writing good user stories: https://cleancommit.io/blog/how-to-write-a-good-user-story-with-examples-templates/ (Nope, it would be still a bad user story, if you just copied my example and added a proper "so that ...". :-) you should come up with an own one.)
Doesn't matter if its version specific or not for that indication and for policy configuration.
Why? Could you elaborate on such out-of-scope criteria?
My concern: an overall undifferentiated score that is based on "today's" status of a repository - how can this tell me anything about a package's version that was built from some source 2 years ago?
PS: I am not trying to steal the topic, I am just interested in the actual applied value and real-world use cases of OSSF ScoreCard in general and in DependencyTrack in particular.
Hi there, to address your latest question
My concern: an overall undifferentiated score that is based on "today's" status of a repository - how can this tell me anything about a package's version that was built from some source 2 years ago?
I could elaborate from my perspective:
As a basic metric for provenance and pedigree, it is better than nothing and can be part of the solution. In contrast to known vulnerabilities of a specific library version, I see it a non-compehensive, high-level assessment of the quality of the development process of this component. This is not perfect, of course and further analyses must be done, such as "Is the use of this component still necessary?". But this approach is
- automatable
- can be compared
- can be tracked over time
Furthermore, the categories of OpenSSF Scorecard are relatively stable. So I could imagine, that individual thresholds of those categories could be configured as a policy and stakeholders can be notified on violations.
In regulated environments, auditors and stakeholders often encourage/require you to address unknown vulnerabilities and re-assess of the risks of used components regularly. And Dependency Track is - in my opinion - the best place for showing such aggregated information regarding dependencies.
As a basic metric for provenance and pedigree, it is better than nothing and can be part of the solution.
Precisely.
From my perspective: In practice, this kind of high-level assessment of the development process quality can help contextualize existing vulnerabilities (think: is this project actively maintained? can I expect a fix to be released? how large is the community? ...) and, especially when coupled with other project metadata, could be highly interesting for use in policies.
While these Scorecard scores are not a comprehensive assessment of the projects security posture, they can serve as additional information that helps with decision-making.
