dependency-track

dependency-track copied to clipboard

Current Behavior:

Loading an SBOM that used to process and find several hundreds of vulnerabilities, now fails to process and creates an error of 'An unexpected error occurred performing a vulnerability analysis task.' Valid_SBOM_CDX.txt

Steps to Reproduce:

Load the attached example SBOM. Note - you will need to change the attached file to a .json, as .json extensions are not allowed to be attached, so I had to change to a .txt to load. Or, you can grab the SBOM from here.

Expected Behavior:

I expect this to process with no errors and find several hundreds of vulnerabilities, like it used to.

Environment:

• Dependency-Track Version: 4.5.0
• Distribution: Docker
• BOM Format & Version: CycloneDX 1.2
• Database Server: PostgreSQL
• Browser: Google Chrome

Additional Details:

dtrack-apiserver_1  | 2022-05-24 16:08:58,932 INFO [BomUploadProcessingTask] Processing CycloneDX BOM uploaded to project: 883c21fe-e7a7-40f0-a4f7-7aaf810e357a
db_1                | 2022-05-24 16:09:06.628 UTC [123] ERROR:  duplicate key value violates unique constraint "REPOSITORY_META_COMPONENT_COMPOUND_IDX"
db_1                | 2022-05-24 16:09:06.628 UTC [123] DETAIL:  Key ("REPOSITORY_TYPE", "NAMESPACE", "NAME")=(MAVEN, org.springframework.boot, spring-boot-starter-tomcat) already exists.
db_1                | 2022-05-24 16:09:06.628 UTC [123] STATEMENT:  INSERT INTO "REPOSITORY_META_COMPONENT" ("LAST_CHECK","LATEST_VERSION","NAME","NAMESPACE","PUBLISHED","REPOSITORY_TYPE") VALUES ($1,$2,$3,$4,$5,$6)
db_1                |   RETURNING *
dtrack-apiserver_1  | 2022-05-24 16:09:06,629 WARN [Persist] Insert of object "org.dependencytrack.model.RepositoryMetaComponent@68469623" using statement "INSERT INTO "REPOSITORY_META_COMPONENT" ("LAST_CHECK","LATEST_VERSION","NAME","NAMESPACE","PUBLISHED","REPOSITORY_TYPE") VALUES (?,?,?,?,?,?)" failed : ERROR: duplicate key value violates unique constraint "REPOSITORY_META_COMPONENT_COMPOUND_IDX"
dtrack-apiserver_1  |   Detail: Key ("REPOSITORY_TYPE", "NAMESPACE", "NAME")=(MAVEN, org.springframework.boot, spring-boot-starter-tomcat) already exists.
dtrack-apiserver_1  | 2022-05-24 16:09:06,639 WARN [General] ExecutionContext closed with active transaction, so rolling back the active transaction
dtrack-apiserver_1  | 2022-05-24 16:09:06,642 ERROR [LoggableUncaughtExceptionHandler] An unknown error occurred in an asynchronous event or notification thread
dtrack-apiserver_1  | javax.jdo.JDODataStoreException: Insert of object "org.dependencytrack.model.RepositoryMetaComponent@68469623" using statement "INSERT INTO "REPOSITORY_META_COMPONENT" ("LAST_CHECK","LATEST_VERSION","NAME","NAMESPACE","PUBLISHED","REPOSITORY_TYPE") VALUES (?,?,?,?,?,?)" failed : ERROR: duplicate key value violates unique constraint "REPOSITORY_META_COMPONENT_COMPOUND_IDX"
dtrack-apiserver_1  |   Detail: Key ("REPOSITORY_TYPE", "NAMESPACE", "NAME")=(MAVEN, org.springframework.boot, spring-boot-starter-tomcat) already exists.
dtrack-apiserver_1  |   at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:542)
dtrack-apiserver_1  |   at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:720)
dtrack-apiserver_1  |   at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:740)
dtrack-apiserver_1  |   at alpine.persistence.AbstractAlpineQueryManager.persist(AbstractAlpineQueryManager.java:427)
dtrack-apiserver_1  |   at org.dependencytrack.persistence.RepositoryQueryManager.synchronizeRepositoryMetaComponent(RepositoryQueryManager.java:197)
dtrack-apiserver_1  |   at org.dependencytrack.persistence.QueryManager.synchronizeRepositoryMetaComponent(QueryManager.java:993)
dtrack-apiserver_1  |   at org.dependencytrack.tasks.repositories.RepositoryMetaAnalyzerTask.analyze(RepositoryMetaAnalyzerTask.java:95)
dtrack-apiserver_1  |   at org.dependencytrack.tasks.repositories.RepositoryMetaAnalyzerTask.inform(RepositoryMetaAnalyzerTask.java:50)
dtrack-apiserver_1  |   at alpine.event.framework.BaseEventService.lambda$publish$0(BaseEventService.java:99)
dtrack-apiserver_1  |   at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
dtrack-apiserver_1  |   at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
dtrack-apiserver_1  |   at java.base/java.lang.Thread.run(Unknown Source)
dtrack-apiserver_1  | Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "REPOSITORY_META_COMPONENT_COMPOUND_IDX"
dtrack-apiserver_1  |   Detail: Key ("REPOSITORY_TYPE", "NAMESPACE", "NAME")=(MAVEN, org.springframework.boot, spring-boot-starter-tomcat) already exists.
dtrack-apiserver_1  |   at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2675)
dtrack-apiserver_1  |   at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2365)
dtrack-apiserver_1  |   at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:355)
dtrack-apiserver_1  |   at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:490)
dtrack-apiserver_1  |   at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:408)
dtrack-apiserver_1  |   at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:167)
dtrack-apiserver_1  |   at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:135)
dtrack-apiserver_1  |   at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
dtrack-apiserver_1  |   at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
dtrack-apiserver_1  |   at org.datanucleus.store.rdbms.ParamLoggingPreparedStatement.executeUpdate(ParamLoggingPreparedStatement.java:393)
dtrack-apiserver_1  |   at org.datanucleus.store.rdbms.SQLController.executeStatementUpdate(SQLController.java:435)
dtrack-apiserver_1  |   at org.datanucleus.store.rdbms.request.InsertRequest.execute(InsertRequest.java:479)
dtrack-apiserver_1  |   at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertObjectInTable(RDBMSPersistenceHandler.java:162)
dtrack-apiserver_1  |   at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertObject(RDBMSPersistenceHandler.java:138)
dtrack-apiserver_1  |   at org.datanucleus.state.StateManagerImpl.internalMakePersistent(StateManagerImpl.java:4569)
dtrack-apiserver_1  |   at org.datanucleus.state.StateManagerImpl.makePersistent(StateManagerImpl.java:4546)
dtrack-apiserver_1  |   at org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:2026)
dtrack-apiserver_1  |   at org.datanucleus.ExecutionContextImpl.persistObjectWork(ExecutionContextImpl.java:1869)
dtrack-apiserver_1  |   at org.datanucleus.ExecutionContextImpl.persistObject(ExecutionContextImpl.java:1724)
dtrack-apiserver_1  |   at org.datanucleus.ExecutionContextThreadedImpl.persistObject(ExecutionContextThreadedImpl.java:219)
dtrack-apiserver_1  |   at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:715)
dtrack-apiserver_1  |   ... 10 common frames omitted
dtrack-apiserver_1  | 2022-05-24 16:09:09,490 INFO [BomUploadProcessingTask] Processing CycloneDX dependency graph for project: 883c21fe-e7a7-40f0-a4f7-7aaf810e357a
dtrack-apiserver_1  | 2022-05-24 16:09:10,367 INFO [BomUploadProcessingTask] Processed 903 components and 0 services uploaded to project 883c21fe-e7a7-40f0-a4f7-7aaf810e357a
dtrack-apiserver_1  | 2022-05-24 16:09:11,345 INFO [InternalAnalysisTask] Starting internal analysis task
dtrack-apiserver_1  | 2022-05-24 16:09:12,964 INFO [InternalAnalysisTask] Internal analysis complete
dtrack-apiserver_1  | 2022-05-24 16:09:12,967 WARN [OssIndexAnalysisTask] An API username or token has not been specified for use with OSS Index. Using anonymous access
dtrack-apiserver_1  | 2022-05-24 16:09:12,967 INFO [OssIndexAnalysisTask] Starting Sonatype OSS Index analysis task
dtrack-apiserver_1  | 2022-05-24 16:09:13,948 ERROR [VulnerabilityAnalysisTask] An unexpected error occurred performing a vulnerability analysis task
dtrack-apiserver_1  | java.lang.NullPointerException: null
dtrack-apiserver_1  |   at org.dependencytrack.parser.ossindex.OssIndexParser.parse(OssIndexParser.java:78)
dtrack-apiserver_1  |   at org.dependencytrack.parser.ossindex.OssIndexParser.parse(OssIndexParser.java:53)
dtrack-apiserver_1  |   at org.dependencytrack.tasks.scanners.OssIndexAnalysisTask.submit(OssIndexAnalysisTask.java:229)
dtrack-apiserver_1  |   at org.dependencytrack.tasks.scanners.OssIndexAnalysisTask.analyze(OssIndexAnalysisTask.java:171)
dtrack-apiserver_1  |   at org.dependencytrack.tasks.scanners.OssIndexAnalysisTask.inform(OssIndexAnalysisTask.java:111)
dtrack-apiserver_1  |   at org.dependencytrack.tasks.VulnerabilityAnalysisTask.performAnalysis(VulnerabilityAnalysisTask.java:153)
dtrack-apiserver_1  |   at org.dependencytrack.tasks.VulnerabilityAnalysisTask.analyzeComponents(VulnerabilityAnalysisTask.java:119)
dtrack-apiserver_1  |   at org.dependencytrack.tasks.VulnerabilityAnalysisTask.inform(VulnerabilityAnalysisTask.java:70)
dtrack-apiserver_1  |   at alpine.event.framework.BaseEventService.lambda$publish$0(BaseEventService.java:99)
dtrack-apiserver_1  |   at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
dtrack-apiserver_1  |   at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
dtrack-apiserver_1  |   at java.base/java.lang.Thread.run(Unknown Source)
dtrack-apiserver_1  | 2022-05-24 16:09:13,954 INFO [PolicyEngine] Evaluating 903 component(s) against applicable policies
dtrack-apiserver_1  | 2022-05-24 16:09:16,657 INFO [PolicyEngine] Policy analysis complete
dtrack-apiserver_1  | 2022-05-24 16:09:16,659 INFO [MetricsUpdateTask] Executing metrics update for project: 883c21fe-e7a7-40f0-a4f7-7aaf810e357a

This is caused by a breaking change in OSS Index as part of their recent migration: https://ossindex.sonatype.org/updates-notice. Dependency-Check has a similar issue with this: https://github.com/jeremylong/DependencyCheck/issues/4527

Essentially, for unauthenticated users, they now include a hint in API responses that suggests there being more vulnerabilities that can be viewed after authentication:

{
    "id": "sonatype-2021-1681",
    "title": "2 vulnerabilities found",
    "description": "2 non-CVE vulnerabilities found. To see more details, please create a free account at https://https://ossindex.sonatype.org/ and request for this information using your registered account",
    "cvssScore": 8.6
}

Which causes DT's parser to stumble.

According to Sonatype, this is only done for the "proprietary data generated by [their] 65+ person research team and [their] Nexus Intelligence engine". Meaning users get more and higher quality data when authenticated.

Sonatype will add a temporary workaround on their side, so this particular error will hopefully be resolved soon.

That being said, we have a todo here as well - namely hardening our parser logic so it doesn't throw NPEs. And we have to ensure that we make users aware of the limitations when using OSS Index without credentials.

As users of Dependency-Track, your best bet is to register for a free OSS Index account and provide your credentials to Dependency-Track. That resolves both the reported issue as well as provides more quality data.

OSS Index has also been resolving some issues with the new feed, so you should find the incidence of these NPEs be greatly reduced. If you are still running into them please let me know and we will resolve the issue as quickly as possible.

:( i ran into this issue this morning (new install) ... when i attempted to create a free account, it gave me an error :( Anyone else having issues creating a free account??