ossf
What are our barriers to adoption?
This question is related to #12
We claim in our goals and purpose that there are barriers to SBOM adoption. We should be more clear about this.
Rather than just claim this is true, we should find a way to determine this with data. Are there existing studies on the topic we can reuse? Is someone willing to conduct a survey with this group?
How can we move this question forward in a scientific manner that isn't made up data.
The volunteers for helping sort this are Tracy, Bunny, Sarah, Randall
@joshbressers I could start a draft PR and start to compile this information.
I can provide some data from the OWASP community. Based on the data we have, I think the barriers are either non-existent or in some cases quite small. Feel free to reach out and I'll provide the data we have - will likely take a few days to pull together.
FYI, I am waiting until September 1st to pull the latest usage data for August.
These are the current download stats for a few different CycloneDX implementations for the month of August 2022.
| Plugin | Ecosystem | DLs (August) | Notes |
|---|---|---|---|
| @cyclonedx/bom | Javascript | 114,206 | 2.2M DLs total |
| @cyclonedx/webpack-plugin | Javascript | 7,859 | |
| @cyclonedx/cyclonedx-library | Javascript | 9,434 | |
| cyclonedx-core-java | Java | 91,859 | |
| cyclonedx-maven-plugin | Java | 64,578 | Popularity: 244/1000 of Maven plugins |
| cyclonedx-python-lib | Python | 434,328 | Critical Project - Top 1% on pypi |
| cyclonedx-bom | Python | 24,607 | |
| cyclonedx-conan | C/C++ | 742 |
Many organizations utilize local repository servers which cache downloads from official sources such as Maven Central or npm.js. Sonatype Nexus Repository and JFrog Artifactory are examples. When used inside an organization, these repo servers download and store artifacts and when builds require an artifact, they resolve them from cache. As a result, actual usage numbers are higher than stated.
In addition, here are some stats provided by Sonatype that measure the amount of usage that OWASP Dependency-Track systems consume from OSS Index. These are "floor" metrics. One tool measured against one source of vulnerability intelligence. Actual adoption and usage will be much higher.
| Month | Requests | Packages |
|---|---|---|
| August | 9,185,062 | 272,307,889 |
| July | 8,842,782 | 286,485,951 |
| June | 8,618,366 | 270,988,042 |
| May | 7,951,451 | 226,531,600 |
| April | 6,950,128 | 201,661,020 |
Notes:
- Dependency-Track will submit up to 100 components in each request. The average number of components in an application according to Sonatype's 2021 State of the Software Supply Chain report is 128 components
- In April we only had the requests number, not packages, but we estimated 202M at the time which is really close
- Since April, we have had a 35% increase in the number of components represented in CycloneDX
- At a minimum, 272M components are represented in CycloneDX every month
With regard to barriers to adoption from a CycloneDX perspective. The availability of quality tooling that produces SBOMs is not an issue. The ease of doing so is not an issue. The CycloneDX Tool Center has ~150 known tools documented, with many commercial and open source projects having adopted it and provided the capability to their users.
Observations (without data)
- There are far fewer tools that consume and analyze SBOMs
- Legacy ecosystems (e.g. C/C++) that have typically manually managed dependencies have not migrated to Conan or other package managers.
- Most organizations I've talked with use SBOMs for internal purposes only (as a best practice)
- Many organizations will not provide SBOMs until required to do so (regulation, contractual, etc)
The best way to determine if there are barriers to adoption is to measure adoption. I don't think there's any dispute that SBOMs have been broadly adopted by the developer community for product vulnerability management purposes. However, in the non-developer community, adoption is very weak indeed. There are two questions to ask:
- How many suppliers are distributing SBOMs on a regular basis to customers?
- How many software-using organizations (which includes probably every organization over 5 people on the planet - and this includes the "business sides" of software developers) are utilizing SBOMs to manage software risks using automation? That is, how many utilize tooling to ingest SBOMs and identify risks due to components of software deployed in their organization? From my experience co-leading the Energy SBOM Proof of Concept under the NTIA, the problem wasn't the suppliers - a number of large suppliers were quite willing to share their SBOMs with customers in the PoC. It was the electric utilities, none of which even wanted to look at the SBOMs. This was last year. I'd love to see the results of a survey that asked these questions in a scientific manner.
I agree with @Tomalrich We should put together a proposal to have the research wing of Linux Foundation conduct information gathering on this topic.
I have a strong suspicion there are substantial barriers to adoption across the board in this space. The people in this small circle have no problems creating and even consuming SBOMs, but many of the non SBOM nerds I speak with are lost on where to even start.
But rather than guess, we should collect real data so we can make proper decisions. We cannot build policy on top of opinion.
Prior to a proposal for LF, which itself would introduce bias, we may want to see the results of the research conducted by PhD students at UNSW Sydney.
https://www.linkedin.com/posts/boming-xia-589819240_software-bill-of-materials-sbom-survey-activity-6962371513568153600-rnwZ?utm_source=linkedin_share&utm_medium=member_desktop_web
https://twitter.com/BomingXia/status/1556607410625605634?s=20&t=Bz8knZQmjo1x6oIeSqmi_A
In terms of research - this document has a ton of info in it about adoption: https://linuxfoundation.org/wp-content/uploads/LFResearch_SBOM_Report_020422.pdf
It may not answer every question we have, but it covers usage quite well.
Tracy, the problem with the LF report is that it doesn't clearly distinguish software producers from software consumers. It certainly shows that producers are utilizing SBOMs; that isn't the question. I would like to see a survey asking how many producers are distributing SBOMs to customers on a regular basis. If there were at least a few producers that answered yes, that would be quite encouraging.
My own "survey results": I know of no medium-to-large software producer that is regularly distributing SBOMs to customers of at least a few of its products (not just a small test or pilot, although there aren't many of those, either), for vulnerability management purposes (about 4 European auto suppliers are producing SBOMs for a few large autos OEMS, for open source license management purposes). If there's anyone who knows of such a producer, I'd love to hear about it.
Tom
Tom Alrich LLC
312-515-8996
From: Tracy Ragan @.> Sent: Tuesday, September 13, 2022 11:08 AM To: ossf/sbom-everywhere @.> Cc: Tom Alrich @.>; Mention @.> Subject: Re: [ossf/sbom-everywhere] What are our barriers to adoption? (Issue #13)
In terms of research - this document has a ton of info in it about adoption: https://linuxfoundation.org/wp-content/uploads/LFResearch_SBOM_Report_020422.pdf
It may not answer every question we have, but it covers usage quite well.
— Reply to this email directly, view it on GitHubhttps://github.com/ossf/sbom-everywhere/issues/13#issuecomment-1245629236, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AVCDY64XZSABGC5BUWSY7T3V6CRHNANCNFSM56XVJF5A. You are receiving this because you were mentioned.Message ID: @.***>
@Tomalrich I agree with your 'survey results' as I have also yet to see any customer requirements which require SBOMs to be distributed as part of a software delivery in order to support vulnerability management activities. I have also not seen any 'policy' which requires that SBOMs are produced (I am in the UK and I think it is still very much still a watching brief to see what happens with the implementation of the US EO) which once defined would probably result in some increase in production.
But what are the barriers to adoption?
From 'my survey' of multiple SBOM generation tools is that they are not consistent. SPDX offers both files and package level defintions. Both are useful but as a customer I am probably more interested in what you have delivered (which I think equates to packages) rather than what it has been produced from (files). The NTIA minimum content for a SBOM might help but I see a major challenge in determining some of the metadata particularly in the consistency of the the identifiication of the supplier; I have also seen some interesting differences in the handling of the version data of a component between different tools. Having two major standards, SPDX and CycloneDX, is probably a good thing as they should drive innovation but there does need to be some agreed consistency of content.
Retrospectively producing SBOMs from binaries (which is still the most common form of software delivery to a customer) is very hard (I have tried!) so the supply chain needs to be driving the production of SBOMs and not customers.
So how can adoption increase? A quick win would be for software vendors to start automatically producing SBOMs as part of their deliveries. I am starting to see a few products where this is happenning as part of a software release for some open source products but it is far from common practice. In the same way that I have advocated to my colleagues to only use open source software which has a clearly defined (and approved) software licence, having a SBOM could become an additional criteria in the selection of a software component.
Thanks, Anthony. There are a number of things holding SBOM adoption back on the consumer side, although on the producer side, they're already being heavily used (which itself is good, of course. Since the whole point of SBOMs is to get producers to produce more secure software).
I formed an informal group of "SBOM industry" leaders this spring, with the stated purpose of identifying barriers to distribution and use of SBOMs and finding solutions to those problems. The first item we addressed was the "naming problem" in the NVD - i.e. the problem that probably only 5% (if that) of products can be located in the NVD, using what should be their proper CPE names. Our solution is now under consideration by MITRE, CISA and NIST, and it is very likely that something will be done on the problem soon (or at least, work will start), since there's general agreement that this alone is a big impediment. Our document is herehttps://owasp.org/assets/files/posts/A%20Proposal%20to%20Operationalize%20Component%20Identification%20for%20Vulnerability%20Management.pdf.
We've now moved on to the aliasing problem, which is also about naming, but is more general than just the NVD. We meet weekly at 1PM Eastern Time (which I know isn't great for you) pm Friday, and if you'd like to participate in our meetings (and receive the meeting notes), let me know.
Tom
Tom Alrich LLC
312-515-8996
From: anthonyharrison @.> Sent: Tuesday, October 25, 2022 3:33 AM To: ossf/sbom-everywhere @.> Cc: Tom Alrich @.>; Mention @.> Subject: Re: [ossf/sbom-everywhere] What are our barriers to adoption? (Issue #13)
@Tomalrichhttps://github.com/Tomalrich I agree with your 'survey results' as I have also yet to see any customer requirements which require SBOMs to be distributed as part of a software delivery in order to support vulnerability management activities. I have also not seen any 'policy' which requires that SBOMs are produced (I am in the UK and I think it is still very much still a watching brief to see what happens with the implementation of the US EO) which once defined would probably result in some increase in production.
But what are the barriers to adoption?
From 'my survey' of multiple SBOM generation tools is that they are not consistent. SPDX offers both files and package level defintions. Both are useful but as a customer I am probably more interested in what you have delivered (which I think equates to packages) rather than what it has been produced from (files). The NTIA minimum content for a SBOM might help but I see a major challenge in determining some of the metadata particularly in the consistency of the the identifiication of the supplier; I have also seen some interesting differences in the handling of the version data of a component between different tools. Having two major standards, SPDX and CycloneDX, is probably a good thing as they should drive innovation but there does need to be some agreed consistency of content.
Retrospectively producing SBOMs from binaries (which is still the most common form of software delivery to a customer) is very hard (I have tried!) so the supply chain needs to be driving the production of SBOMs and not customers.
So how can adoption increase? A quick win would be for software vendors to start automatically producing SBOMs as part of their deliveries. I am starting to see a few products where this is happenning as part of a software release for some open source products but it is far from common practice. In the same way that I have advocated to my colleagues to only use open source software which has a clearly defined (and approved) software licence, having a SBOM could become an additional criteria in the selection of a software component.
— Reply to this email directly, view it on GitHubhttps://github.com/ossf/sbom-everywhere/issues/13#issuecomment-1290188475, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AVCDY64DHE7SBSNVWBTDV63WE6LO7ANCNFSM56XVJF5A. You are receiving this because you were mentioned.Message ID: @.***>
Tom
Thanks for the feedback. It is good to see that some of the challenges with SBOMs are starting to be addressed.
Although the timing isn't great to attend your meetings, I would like to be involved (I might be able to make some of the meetings depending on personal circumstances) and kept informed of progress. IAs I am developing a number of SBOM related tools it would be good to try and align these with the emerging best practices.
Kind regards
Anthony
On Tue, 25 Oct 2022 at 12:00, Tomalrich @.***> wrote:
Thanks, Anthony. There are a number of things holding SBOM adoption back on the consumer side, although on the producer side, they're already being heavily used (which itself is good, of course. Since the whole point of SBOMs is to get producers to produce more secure software).
I formed an informal group of "SBOM industry" leaders this spring, with the stated purpose of identifying barriers to distribution and use of SBOMs and finding solutions to those problems. The first item we addressed was the "naming problem" in the NVD - i.e. the problem that probably only 5% (if that) of products can be located in the NVD, using what should be their proper CPE names. Our solution is now under consideration by MITRE, CISA and NIST, and it is very likely that something will be done on the problem soon (or at least, work will start), since there's general agreement that this alone is a big impediment. Our document is here< https://owasp.org/assets/files/posts/A%20Proposal%20to%20Operationalize%20Component%20Identification%20for%20Vulnerability%20Management.pdf
.
We've now moved on to the aliasing problem, which is also about naming, but is more general than just the NVD. We meet weekly at 1PM Eastern Time (which I know isn't great for you) pm Friday, and if you'd like to participate in our meetings (and receive the meeting notes), let me know.
Tom
Tom Alrich LLC
312-515-8996
From: anthonyharrison @.> Sent: Tuesday, October 25, 2022 3:33 AM To: ossf/sbom-everywhere @.> Cc: Tom Alrich @.>; Mention @.> Subject: Re: [ossf/sbom-everywhere] What are our barriers to adoption? (Issue #13)
@Tomalrichhttps://github.com/Tomalrich I agree with your 'survey results' as I have also yet to see any customer requirements which require SBOMs to be distributed as part of a software delivery in order to support vulnerability management activities. I have also not seen any 'policy' which requires that SBOMs are produced (I am in the UK and I think it is still very much still a watching brief to see what happens with the implementation of the US EO) which once defined would probably result in some increase in production.
But what are the barriers to adoption?
From 'my survey' of multiple SBOM generation tools is that they are not consistent. SPDX offers both files and package level defintions. Both are useful but as a customer I am probably more interested in what you have delivered (which I think equates to packages) rather than what it has been produced from (files). The NTIA minimum content for a SBOM might help but I see a major challenge in determining some of the metadata particularly in the consistency of the the identifiication of the supplier; I have also seen some interesting differences in the handling of the version data of a component between different tools. Having two major standards, SPDX and CycloneDX, is probably a good thing as they should drive innovation but there does need to be some agreed consistency of content.
Retrospectively producing SBOMs from binaries (which is still the most common form of software delivery to a customer) is very hard (I have tried!) so the supply chain needs to be driving the production of SBOMs and not customers.
So how can adoption increase? A quick win would be for software vendors to start automatically producing SBOMs as part of their deliveries. I am starting to see a few products where this is happenning as part of a software release for some open source products but it is far from common practice. In the same way that I have advocated to my colleagues to only use open source software which has a clearly defined (and approved) software licence, having a SBOM could become an additional criteria in the selection of a software component.
— Reply to this email directly, view it on GitHub< https://github.com/ossf/sbom-everywhere/issues/13#issuecomment-1290188475>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AVCDY64DHE7SBSNVWBTDV63WE6LO7ANCNFSM56XVJF5A
. You are receiving this because you were mentioned.Message ID: @.***>
— Reply to this email directly, view it on GitHub https://github.com/ossf/sbom-everywhere/issues/13#issuecomment-1290361786, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACAID22SZUWBPRO4N7A63IDWE64THANCNFSM56XVJF5A . You are receiving this because you commented.Message ID: @.***>
Thanks, Anthony. Please send me your email address. You can send it to @.***
Tom Alrich LLC
312-515-8996
From: anthonyharrison @.> Sent: Wednesday, October 26, 2022 4:26 AM To: ossf/sbom-everywhere @.> Cc: Tom Alrich @.>; Mention @.> Subject: Re: [ossf/sbom-everywhere] What are our barriers to adoption? (Issue #13)
Tom
Thanks for the feedback. It is good to see that some of the challenges with SBOMs are starting to be addressed.
Although the timing isn't great to attend your meetings, I would like to be involved (I might be able to make some of the meetings depending on personal circumstances) and kept informed of progress. IAs I am developing a number of SBOM related tools it would be good to try and align these with the emerging best practices.
Kind regards
Anthony
On Tue, 25 Oct 2022 at 12:00, Tomalrich @.***> wrote:
Thanks, Anthony. There are a number of things holding SBOM adoption back on the consumer side, although on the producer side, they're already being heavily used (which itself is good, of course. Since the whole point of SBOMs is to get producers to produce more secure software).
I formed an informal group of "SBOM industry" leaders this spring, with the stated purpose of identifying barriers to distribution and use of SBOMs and finding solutions to those problems. The first item we addressed was the "naming problem" in the NVD - i.e. the problem that probably only 5% (if that) of products can be located in the NVD, using what should be their proper CPE names. Our solution is now under consideration by MITRE, CISA and NIST, and it is very likely that something will be done on the problem soon (or at least, work will start), since there's general agreement that this alone is a big impediment. Our document is here< https://owasp.org/assets/files/posts/A%20Proposal%20to%20Operationalize%20Component%20Identification%20for%20Vulnerability%20Management.pdf
.
We've now moved on to the aliasing problem, which is also about naming, but is more general than just the NVD. We meet weekly at 1PM Eastern Time (which I know isn't great for you) pm Friday, and if you'd like to participate in our meetings (and receive the meeting notes), let me know.
Tom
Tom Alrich LLC
312-515-8996
From: anthonyharrison @.> Sent: Tuesday, October 25, 2022 3:33 AM To: ossf/sbom-everywhere @.> Cc: Tom Alrich @.>; Mention @.> Subject: Re: [ossf/sbom-everywhere] What are our barriers to adoption? (Issue #13)
@Tomalrichhttps://github.com/Tomalrich I agree with your 'survey results' as I have also yet to see any customer requirements which require SBOMs to be distributed as part of a software delivery in order to support vulnerability management activities. I have also not seen any 'policy' which requires that SBOMs are produced (I am in the UK and I think it is still very much still a watching brief to see what happens with the implementation of the US EO) which once defined would probably result in some increase in production.
But what are the barriers to adoption?
From 'my survey' of multiple SBOM generation tools is that they are not consistent. SPDX offers both files and package level defintions. Both are useful but as a customer I am probably more interested in what you have delivered (which I think equates to packages) rather than what it has been produced from (files). The NTIA minimum content for a SBOM might help but I see a major challenge in determining some of the metadata particularly in the consistency of the the identifiication of the supplier; I have also seen some interesting differences in the handling of the version data of a component between different tools. Having two major standards, SPDX and CycloneDX, is probably a good thing as they should drive innovation but there does need to be some agreed consistency of content.
Retrospectively producing SBOMs from binaries (which is still the most common form of software delivery to a customer) is very hard (I have tried!) so the supply chain needs to be driving the production of SBOMs and not customers.
So how can adoption increase? A quick win would be for software vendors to start automatically producing SBOMs as part of their deliveries. I am starting to see a few products where this is happenning as part of a software release for some open source products but it is far from common practice. In the same way that I have advocated to my colleagues to only use open source software which has a clearly defined (and approved) software licence, having a SBOM could become an additional criteria in the selection of a software component.
— Reply to this email directly, view it on GitHub< https://github.com/ossf/sbom-everywhere/issues/13#issuecomment-1290188475>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AVCDY64DHE7SBSNVWBTDV63WE6LO7ANCNFSM56XVJF5A
. You are receiving this because you were mentioned.Message ID: @.***>
— Reply to this email directly, view it on GitHub https://github.com/ossf/sbom-everywhere/issues/13#issuecomment-1290361786, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACAID22SZUWBPRO4N7A63IDWE64THANCNFSM56XVJF5A . You are receiving this because you commented.Message ID: @.***>
— Reply to this email directly, view it on GitHubhttps://github.com/ossf/sbom-everywhere/issues/13#issuecomment-1291755679, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AVCDY62YYCD4LEMTSTSCP7LWFD2OHANCNFSM56XVJF5A. You are receiving this because you were mentioned.Message ID: @.***>