ossf
✨ Add Check for published SBOM
What kind of change does this PR introduce?
Adds check for published sboms. This PR is still a draft as there is more discussion on applicability and implementation to be had. This PR is meant to spur those conversations.
- [x] PR title follows the guidelines defined in our pull request documentation
What is the current behavior?
N/A
What is the new behavior (if this is a feature change)?**
*Tests were not added as the implementation may change, tests will be added prior to merge.
Adds a multi probe check for a Software Bill of Materials for a scanned repository.
More information regarding implementation and considerations for this check can be found here
Which issue(s) this PR fixes
Fixes #3574, #1476
Special notes for your reviewer
Does this PR introduce a user-facing change?
For user-facing changes, please add a concise, human-readable release note to
the release-note
(In particular, describe what changes users might need to make in their application as a result of this pull request.)
Codecov Report
Merging #3903 (037eb1a) into main (d58bfb0) will decrease coverage by
4.67%. Report is 1 commits behind head on main. The diff coverage is67.73%.
Additional details and impacted files
@@ Coverage Diff @@
## main #3903 +/- ##
==========================================
- Coverage 74.94% 70.28% -4.67%
==========================================
Files 223 232 +9
Lines 16046 16723 +677
==========================================
- Hits 12026 11754 -272
- Misses 3253 4230 +977
+ Partials 767 739 -28
![codecov[bot] avatar](https://avatars.githubusercontent.com/in/254?v=4 "Published by a pub.dev verified publisher"){kind=small}
@laurentsimon
- Should the probe be aware of cases where there's no release assets? For example, a GitHub release may have no assets, if it's only source code release. Is an SBOM needed in this case? Note that the release could contain a container release as a GitHub package or something else. We could (maybe?) update the code to detect the former case, but the latter seems much harder.
Both clients as written handle the possibility that there are no release assets, in which case it just continues on with the remainder of the check. For a source only release I'd expect an associated SBOM, as ideally each released version would have an associated SBOM for downstream users (assuming the project should be creating an SBOM).
For releases that include a container, we could potentially check for an SBOM layer in the container, but the effort involved in that may not be worth it. This would be a situation where the workflow that builds the container in preparation for release would ideally also generate an SBOM for it and save it as an artifact (at the very least) to be picked up by our checks, and to be available for downstream users.
- How do we determine if the project should create an SBOM or not, depending on the type of release (application, library, ?) - see https://blog.deps.dev/zillions-of-sboms/. Is there a document describing when an SBOM makes sense in each ecosystem?
I'm not sure about this. I know applicability is one of the sticking points for this check in general. There was talk of a Maintainers Annotation feature which could help here, but I'm not sure the status of that.
I'm unaware of any such document describing when an SBOM makes sense in each ecosystem. @idunbarh do you have any insight on this?
- Since GH has an API to generate SBOMs, why should maintainers generate an SBOM themselves?
This I actually forgot to mention in the notes doc I posted, I have since added it. I do hit the SBOM api endpoint for Github repos and, assuming a good response, count that generated SBOM as a release artifact and award appropriate points. As far as generating an SBOM themselves in GH repos, I would leave it up to the maintainers to decide if that is necessary for their project. i.e. if they need to generate one for a release container or in another format. I'm hesitant to award full points for the endpoint BOM, as it may not be comprehensive. I haven't seen anything from GL yet, but would expect to see a similar SBOM endpoint eventually.
I'm unaware of any such document describing when an SBOM makes sense in each ecosystem. @idunbarh do you have any insight on this?
The I don't believe the OpenSSF SBOM Everywhere SIG has provided any guidance on what ecosystems should or should not generate SBOMs. The SBOM Naming Convention Doc would be the ideal place for clarification.
I'm unaware of any such document describing when an SBOM makes sense in each ecosystem. @idunbarh do you have any insight on this?
The I don't believe the OpenSSF SBOM Everywhere SIG has provided any guidance on what ecosystems should or should not generate SBOMs. The SBOM Naming Convention Doc would be the ideal place for clarification.
Yeah we really need a doc on this, per ecosystem, per-use case. What we found in https://blog.deps.dev/zillions-of-sboms/, is that most open-source projects are libraries, so often times an SBOM is not needed (there are nuances, of course). Keep me in the loop if this works starts.
I'm hesitant to award full points for the endpoint BOM, as it may not be comprehensive
For a first integration, one possibility could be to not include these probes in a default check. Users who want to detect the presence of SBOMs in a repo could then run the probes / checks they care about, using the context and knowledge they have about the repo. For example, organizations (@UlisesGascon @netomi) could use their own policy to decide whether to run this probe or not on their projects.
For arbitrary projects, I think we need more investigation to reduce noise (false positives, score decrease)
Yeah we really need a doc on this, per ecosystem, per-use case. What we found in https://blog.deps.dev/zillions-of-sboms/, is that most open-source projects are libraries, so often times an SBOM is not needed (there are nuances, of course).
Yeah, I was going to bring this up. Most projects that'll ever be handled by Scorecard likely don't need an SBOM (simple libraries).
For a first integration, one possibility could be to not include these probes in a default check.
Alternatively, make this a "bonus points" check: 10/10 if the latest release has an SBOM, $INCONCLUSIVE otherwise.
Coming from the Security Tooling WG discussions, one of the desired outcomes is to measure the impact other parts of OpenSSF are having around SBOM adoption. In this case the Security Tooling WG's SBOM Naming Conventions and future SBOM Strike Force that will be working directly with large projects to implement SBOMs. I see Scorecard as an mechanism to help adoption and also measure adoption.
I'm all for making it a bonus points check over a check that would be disabled by default.
Coming from the Security Tooling WG discussions, one of the desired outcomes is to measure the impact other parts of OpenSSF are having around SBOM adoption. In this case the Security Tooling WG's SBOM Naming Conventions and future SBOM Strike Force that will be working directly with large projects to implement SBOMs. I see Scorecard as an mechanism to help adoption and also measure adoption.
That's totally fine. But let's not lose sight of the objective, which is to increase the security of producing and consuming projects. We must be careful not to require an SBOM when none is needed. If we don't do that, it will be detrimental to OSSF and SBOMs: add noise and unnecessary work to developers, generate inaccurate SBOM, etc.. which will eventually decrease adoption and credibility.
Alternatively, make this a "bonus points" check: 10/10 if the latest release has an SBOM, $INCONCLUSIVE otherwise.
A point will be interpreted by users as something they need to do, regardless of whether we call it bonus or not. As soon as we want to "encourage" via default checks, the check must be low false positive and remediation must be crisp to avoid sending users down a rabbit hole and waste their time. Until we have a good understanding of that, imo it's better to let organizations decide when they require it and give them the ability to enable the check if they want to. That fits well with the first goal of "will be working directly with large projects to implement SBOMs". This also gives us room to improve and adjust and learn from these first engagements.
@JoelMarcey is this something you intend to build for npm / pypi / homebrew / rust?
SBOMs for the Rust ecosystem, particularly around crates.io, is on our radar, but at this point not our highest priority. The Rust Secure code working group [1] is looking at this, Microsoft has a concerted effort happening [2] and there are other tools [3] and protocols [4] out there. But there is still a lot of questions in the air on how best to generate SBOMs. What is the standard? We have cargo.lock files which provide some provenance information, but doesn’t follow any standard format. I think until it is settled on the format that will be used, hopefully across ecosystems, we are in some ways in a holding pattern.
cc @walterhpearce
[1] https://www.rust-lang.org/governance/wgs/wg-secure-code [2] https://github.com/microsoft/sbom-tool [3] https://docs.rs/crate/cargo-sbom/latest [4] https://omnibor.io/
To extend what @JoelMarcey said:
We've spent months revamping cyclonedx-rust-cargo as part of a funded project by the german Sovereign Tech Fund. This tool generates CycloneDX SBOMs for Rust projects. We wrote a blog post about the first part of the project but not yet about the currently running second part.
We are currently writing a PoC for the RFC around having native "SBOM support-supprort" in Rust itself. This is coordinated with @arlosi from Microsoft who started the RFC based on an initial conversation we had around this last year. As part of this RFC we (referring to the Rust ecosystem/contributors) are trying to add the things needed to generate accurate SBOMs to Rust/Cargo itself. It will probably be format agnostic but will have all the information needed for downstream tools, hence it will be able to "support" downstream SBOM generators.
This is a direct result of our work on the CycloneDX generator where we saw which gaps there currently are (e.g. parsing Cargo.toml alone is not sufficient, neither is cargo.lock so we currently call cargo metadata but that is also missing bits and pieces).
I obviously hope that crates.io and other tools will be able to benefit from this when this is done.
And with cargo-auditable we have a tool to embed SBOMs in generated binary artifacts directly and as part of the project we extended it to write CycloneDX as well.
All of this is very high priority here in Europe due to the upcoming Cyber Resilience Act which will make SBOMs more or less mandatory for every digital product as of 2027 or so (depending on when they finally sign it).
@JoelMarcey is this something you intend to build for npm / pypi / homebrew / rust?
In short: Stuff is happening for the Rust ecosystem at least.
A point will be interpreted by users as something they need to do, regardless of whether we call it bonus or not. As soon as we want to "encourage" via default checks, the check must be low false positive and remediation must be crisp to avoid sending users down a rabbit hole and waste their time. Until we have a good understanding of that, imo it's better to let organizations decide when they require it and give them the ability to enable the check if they want to.
I want to ++ this: I think many (especially smaller) projects already struggle to consume scores and treat them as a faulty dimension reduction of the project's actual security posture, so giving them an additional negative weight based on the absence of data rather than the presence of negative data will probably make it harder to justify adoption 🙂
This pull request has been marked stale because it has been open for 10 days with no activity
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
The probe code is in a good spot. You'll also need to add the raw data collection to support running --probes hasSBOM (couldn't comment in the review because the file wasnt modified):
https://github.com/ossf/scorecard/blob/c11d89bfe666eb72c3e83c13b5c82a0c2453589a/pkg/scorecard_result.go#L250-L251
After trying on github.com/bomctl/bomctl, the output can be pretty verbose. Maybe we should limit release lookback to 5 most recent similar to signed releases?
Click to expand
SCORECARD_EXPERIMENTAL=1 go run main.go --repo github.com/bomctl/bomctl --format json --show-details --checks sbom
| jq
{
"date": "2024-05-15T12:22:25-07:00",
"repo": {
"name": "github.com/bomctl/bomctl",
"commit": "4ef4b33ce10500a853403066e335a1d48e838b11"
},
"scorecard": {
"version": "devel",
"commit": "unknown"
},
"score": 10.0,
"checks": [
{
"details": [
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175727:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175726:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175730:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175729:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175723:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175722:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175721:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175720:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175725:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175724:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175719:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175718:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162321:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162322:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162317:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162316:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162323:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162324:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162325:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162327:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162320:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162319:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162315:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162314:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865640:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865638:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865631:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865635:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865637:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865636:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865629:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865630:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865644:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865643:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865642:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865641:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457982:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457986:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457981:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457983:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457989:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457990:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457978:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457979:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457987:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457988:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457975:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457977:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454378:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454379:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454371:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454372:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454381:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454380:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454375:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454376:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454387:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454384:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454383:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454382:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163283690:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163283686:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163283691:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163283688:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163283689:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163283687:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163123197:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163123201:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163123198:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163123200:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163123196:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163123195:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163122640:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163122637:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163122638:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163122636:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163122641:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163122635:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/162076784:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/162076780:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/162076782:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/162076783:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/162076781:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/162076778:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/153684271:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/153684275:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/153684270:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/153684272:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/153684269:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/153684273:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175727:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175726:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175730:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175729:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175723:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175722:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175721:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175720:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175725:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175724:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175719:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175718:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162321:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162322:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162317:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162316:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162323:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162324:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162325:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162327:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162320:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162319:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162315:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162314:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865640:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865638:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865631:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865635:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865637:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865636:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865629:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865630:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865644:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865643:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865642:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865641:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457982:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457986:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457981:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457983:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457989:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457990:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457978:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457979:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457987:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457988:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457975:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457977:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454378:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454379:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454371:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454372:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454381:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454380:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454375:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454376:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454387:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454384:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454383:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454382:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163283690:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163283686:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163283691:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163283688:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163283689:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163283687:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163123197:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163123201:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163123198:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163123200:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163123196:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163123195:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163122640:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163122637:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163122638:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163122636:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163122641:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163122635:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/162076784:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/162076780:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/162076782:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/162076783:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/162076781:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/162076778:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/153684271:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/153684275:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/153684270:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/153684272:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/153684269:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/153684273:0"
],
"score": 10,
"reason": "SBOM file found in release artifacts",
"name": "SBOM",
"documentation": {
"url": "https://github.com/ossf/scorecard/blob/main/docs/checks.md#sbom",
"short": "Determines if the project maintains a Software Bill of Materials."
}
}
],
"metadata": null
}
Same cmd with latest update. I changed it to check the 5 latest releases and only iterate over release assets until a match was found. So puts a cap on release sboms returned to 5
$ SCORECARD_EXPERIMENTAL=1 go run main.go --repo github.com/bomctl/bomctl --format json --show-details --checks sbom | jq
{
"date": "2024-05-15T18:32:00-06:00",
"repo": {
"name": "github.com/bomctl/bomctl",
"commit": "4ef4b33ce10500a853403066e335a1d48e838b11"
},
"scorecard": {
"version": "devel",
"commit": "unknown"
},
"score": 10.0,
"checks": [
{
"details": [
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175727:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162321:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865640:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457982:0",
"Info: Project has a SBOM file: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454378:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165175727:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/165162321:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163865640:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163457982:0",
"Info: Project publishes an SBOM file as part of a release or CICD: https://api.github.com/repos/bomctl/bomctl/releases/assets/163454378:0"
],
"score": 10,
"reason": "SBOM file found in release artifacts",
"name": "SBOM",
"documentation": {
"url": "https://github.com/ossf/scorecard/blob/main/docs/checks.md#sbom",
"short": "Determines if the project maintains a Software Bill of Materials."
}
}
],
"metadata": null
}
gating the check behind experimental flag
Can I ask where this is going to be used? I suspect half-baked SBOMs created to make scorecard happy combined with bogus vulnerabilities like https://github.com/google/osv.dev/issues/2176 are going to waste even more maintainers' time
This PR is still a draft as there is more discussion on applicability and implementation to be had
Given that it's no longer a draft I assume it was discussed and okayed somewhere. Is it possible to take a look at those discussions anywhere?
Can I ask where this is going to be used? I suspect half-baked SBOMs created to make scorecard happy combined with bogus vulnerabilities like google/osv.dev#2176 are going to waste even more maintainers' time
by people who run the scorecard binary on a repo and explicitly mention they want the SBOM check/probe.
This PR is still a draft as there is more discussion on applicability and implementation to be had
Given that it's no longer a draft I assume it was discussed and okayed somewhere. Is it possible to take a look at those discussions anywhere?
This was discussed during OSS NA 2024, there was some talk about it in the OpenSSF Slack as well. I've included some of the relevant Slack discussion below since slack history will hide it soon:
slack conversation
Dana Wang: I met with Spencer Schrock, Laurent Simon, Ian Dunbar-Hall, Allan Friedman, Ryan Ware, justaugustus today on adding a non-opinionated check to Scorecard to report whether a project produces SBOM as the first "gold star" (Ian Dunbar-Hall owns the copyright) to report the existence of SBOM, it helps to gain visibility into the SBOM state, and put zero burden on maintainers, I feel it's a good balance on pushing the boundary on SBOM which is coming with acceleration, and protecting our maintainers. Ryan Ware has a great idea on how the Scorecard score could be tiered/applied to a project to incrementally to enhance the security posture of an open source project while protecting our maintainers from being overwhelmed with security burdens.
Brandon Mitchell: I find myself triggered by the phrase "zero burden on maintainers". I really don't see how that's possible. Is there an example of a public project where this has been done without any maintainer involvement?
Spencer Schrock For context, we're currently envisioning this as something that lives as a probe, not in a check. So it would not be a check in the sense of a Scorecard check, but rather as a binary "is there an SBOM". So there would be no score attached in the normal output / score, just a "is there an SBOM?" "yes/no" when using the --format probe option.
Spencer Schrock We're hoping this allows individual consumers to find out what their dependencies are doing, and if it matches their policy. And it may not, so the end result is "company X shouldn't use this dependency.", not that project Y should start doing Z for the benefit of company X
The consensus wasn't around including it as a default check, but rather getting some code in there for the purpose of:
- Individuals being able to run the analysis if they want it (either through
--checks SBOMor the corresponding--probes hasSBOM,hasReleaseSBOM) - Possibly running it as part of the weekly cron for the purpose of seeing how widely adopted SBOMs are currently. Not for the purpose of assigning a score.
As you can guess, the "default" aspect of Scorecard is the controversial part. This applies to other checks/probes as well. For example some people care about permissive license (#3838), but there's nothing inherently right or wrong with permissive vs copyleft vs other.
I've written more on this topic (which I'll be sharing "soon"), but ultimately Scorecard is a linter and should support any relevant analyses and move away from all of them being in the default output, but rather used through the enabling/disabling with individual configuration. Which lets different audiences use the tool in the manner they want without anything forced upon them.
Got it. Thank you!
people who run the scorecard binary on a repo and explicitly mention they want the SBOM check/probe
I get that part but in https://github.com/ossf/scorecard/issues/3574 (and in some other issues related to SBOMs) I tried to figure out what the use cases were and I haven't seen any actual use cases anywhere.
Individuals being able to run the analysis if they want it
I wonder why? I assume this data is supposed to be acted on.
Not for the purpose of assigning a score
That's reassuring but I'm not talking about scores. I'm kind of concerned that things are seemingly getting adopted with no clear purpose and combined with OSV it offloads a lot of things onto upstream projects without any scores involved.
ultimately Scorecard is a linter and should support any relevant analyses and move away from all of them being in the default output
I agree with that.
🥳 Thanks @spencerschrock for all of the guidance and feedback through this PR!