DependencyCheck
DependencyCheck copied to clipboard
jeremylong
NPE from OssIndexAnalyzer fetching component-report
Describe the bug
[WARNING] Failed to fetch component-report for: pkg:maven/io.netty/[email protected]
java.lang.NullPointerException
at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.transform (OssIndexAnalyzer.java:350)
at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.lambda$enrich$5 (OssIndexAnalyzer.java:256)
at java.util.stream.ReferencePipeline$3$1.accept (ReferencePipeline.java:195)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining (ArrayList.java:1655)
at java.util.stream.AbstractPipeline.copyInto (AbstractPipeline.java:484)
at java.util.stream.AbstractPipeline.wrapAndCopyInto (AbstractPipeline.java:474)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential (ForEachOps.java:150)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential (ForEachOps.java:173)
at java.util.stream.AbstractPipeline.evaluate (AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEachOrdered (ReferencePipeline.java:502)
at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.enrich (OssIndexAnalyzer.java:257)
at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.analyzeDependency (OssIndexAnalyzer.java:163)
at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze (AbstractAnalyzer.java:131)
at org.owasp.dependencycheck.AnalysisTask.call (AnalysisTask.java:88)
at org.owasp.dependencycheck.AnalysisTask.call (AnalysisTask.java:37)
at java.util.concurrent.FutureTask.run (FutureTask.java:264)
at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128)
at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
at java.lang.Thread.run (Thread.java:829)
Version of dependency-check used dependency-check-maven 7.1.0
Log file https://gist.github.com/OrangeDog/9f976a83dd5af51f51ce677944384420
To Reproduce
Unsure. netty-handler is included via io.lettuce:lettuce-core:6.1.8.RELEASE
Expected behavior No NPE warnings in log.
In another project I get the same NPEs for the following:
pkg:maven/commons-httpclient/[email protected]
pkg:maven/com.fasterxml.woodstox/[email protected]
pkg:maven/org.thymeleaf.extras/[email protected]
Same here for the following in our project:
pkg:maven/org.bouncycastle/[email protected]
pkg:maven/io.github.classgraph/[email protected]
pkg:maven/org.bouncycastle/[email protected]
pkg:maven/io.netty/[email protected]
Same for
pkg:maven/com.h2database/[email protected]
pkg:maven/com.google.guava/[email protected]
pkg:maven/com.google.guava/[email protected]
pkg:maven/org.bouncycastle/[email protected]
And
17:03:04 [INFO] --- dependency-check-maven:7.1.0:aggregate (default-cli) @ discovery ---
17:03:06 [INFO] Checking for updates
17:03:06 [INFO] Skipping NVD check since last check was within 4 hours.
17:03:06 [INFO] Skipping RetireJS update since last update was within 24 hours.
17:03:06 [INFO] Check for updates complete (12 ms)
...
17:03:06 [INFO] Analysis Started
17:03:07 [INFO] Finished Archive Analyzer (0 seconds)
17:03:07 [INFO] Finished File Name Analyzer (0 seconds)
17:03:07 [INFO] Finished Jar Analyzer (0 seconds)
17:03:07 [INFO] Finished Dependency Merging Analyzer (0 seconds)
17:03:07 [INFO] Finished Version Filter Analyzer (0 seconds)
17:03:07 [INFO] Finished Hint Analyzer (0 seconds)
17:03:08 [INFO] Created CPE Index (1 seconds)
17:03:10 [INFO] Finished CPE Analyzer (2 seconds)
17:03:10 [INFO] Finished False Positive Analyzer (0 seconds)
17:03:10 [INFO] Finished NVD CVE Analyzer (0 seconds)
17:03:10 00:00 INFO: Vulnerability found: bootstrap below 3.4.1
17:03:10 00:00 INFO: Vulnerability found: bootstrap below 3.4.0
17:03:10 00:00 INFO: Vulnerability found: bootstrap below 3.4.0
17:03:10 00:00 INFO: Vulnerability found: bootstrap below 3.4.0
17:03:10 00:00 INFO: Vulnerability found: bootstrap below 3.4.0
17:03:10 [INFO] Finished RetireJS Analyzer (0 seconds)
17:03:10 [WARNING] Failed to fetch component-report for: pkg:maven/org.bouncycastle/[email protected]
17:03:10 java.lang.NullPointerException: Cannot invoke "java.net.URI.toString()" because the return value of "org.sonatype.ossindex.service.api.componentreport.ComponentReportVulnerability.getReference()" is null
17:03:10 at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.transform (OssIndexAnalyzer.java:350)
17:03:10 at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.lambda$enrich$5 (OssIndexAnalyzer.java:256)
...
17:03:10 [WARNING] Failed to fetch component-report for: pkg:maven/com.google.guava/[email protected]
17:03:10 java.lang.NullPointerException: Cannot invoke "java.net.URI.toString()" because the return value of "org.sonatype.ossindex.service.api.componentreport.ComponentReportVulnerability.getReference()" is null
17:03:10 at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.transform (OssIndexAnalyzer.java:350)
17:03:10 at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.lambda$enrich$5 (OssIndexAnalyzer.java:256)
...
17:03:10 [INFO] Finished Sonatype OSS Index Analyzer (0 seconds)
17:03:10 [INFO] Finished Vulnerability Suppression Analyzer (0 seconds)
17:03:10 [INFO] Finished Dependency Bundling Analyzer (0 seconds)
17:03:10 [INFO] Analysis Complete (4 seconds)
Same here. OSS Index is being upgraded, and it looks like it's starting to report new vulnerabilities that are not CVEs (you need to be registered to see the details of these new types of vulnerabilities).
I tested their REST API with the following payload:
{
"coordinates": [
"pkg:maven/org.terracotta/[email protected]"
]
}
and got the following response:
[
{
"coordinates": "pkg:maven/org.terracotta/[email protected]",
"description": "A library that offers data structures allocated off the java heap.",
"reference": "https://ossindex.sonatype.org/component/pkg:maven/org.terracotta/[email protected]?utm_source=mozilla&utm_medium=integration&utm_content=5.0",
"vulnerabilities": [
{
"id": "sonatype-2020-0267",
"title": "1 vulnerability found",
"description": "1 non-CVE vulnerability found. To see more details, please create a free account at https://https://ossindex.sonatype.org/ and request for this information using your registered account",
"cvssScore": 5.9
}
],
"sonatypeOssiScore": 5.9
}
]
It seems to be what's confusing dependency-check.
@bjansen that would also explain all the false positives that have appeared at the same time. The OSS Index appears to have a bunch of new CPE/version errors.
we are getting the same
Failed to fetch component-report for: pkg:maven/com.fasterxml.jackson.core/[email protected] java.lang.NullPointerException at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.transform(OssIndexAnalyzer.java:350) at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.lambda$enrich$5(OssIndexAnalyzer.java:25
OSS Index is now filtering "sonatype-id" vulnerability notifications for anonymous users. This may only be a partial workaround, as the "sonatype-id" vulnerability details will be visible for authenticating users.
I would like to work around this temporarily by just turning off the OSS Index analyser entirely.
I thought I would be able to achieve this (Maven plugin here) by:
<plugin>
<groupId>org.owasp</groupId>
<artifactId>dependency-check-maven</artifactId>
<version>7.1.0</version>
<configuration>
<!-- Turn off the OSS Index analyser. Currently causing problems with format change and rate limiting.
See assorted Github issues e.g. https://github.com/jeremylong/DependencyCheck/issues/4527. -->
<ossindexAnalyzerEnabled>false</ossindexAnalyzerEnabled>
</configuration>
</plugin>
But this isn't working as I expected. The analyser is still running and throwing out the errors as documented by others in this issue. What am I missing? How do I just entirely disable the OSS Index analyser if not this way?
Hello, I am using the "org.owasp.dependencycheck" gradle plugin, version "7.1.0.1" and I am getting the same issue but for different (transitive) dependencies:
Failed to fetch component-report for: pkg:maven/xerces/[email protected]
java.lang.NullPointerException
at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.transform(OssIndexAnalyzer.java:350)
at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.lambda$enrich$5(OssIndexAnalyzer.java:256)
Failed to fetch component-report for: pkg:maven/com.h2database/[email protected]
java.lang.NullPointerException
at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.transform(OssIndexAnalyzer.java:350)
at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.lambda$enrich$5(OssIndexAnalyzer.java:256)
Failed to fetch component-report for: pkg:maven/org.bouncycastle/[email protected]
java.lang.NullPointerException
at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.transform(OssIndexAnalyzer.java:350)
at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.lambda$enrich$5(OssIndexAnalyzer.java:256)
Failed to fetch component-report for: pkg:maven/javax.mail/[email protected]
java.lang.NullPointerException
at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.transform(OssIndexAnalyzer.java:350)
at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.lambda$enrich$5(OssIndexAnalyzer.java:256)
Failed to fetch component-report for: pkg:maven/io.netty/[email protected]
java.lang.NullPointerException
at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.transform(OssIndexAnalyzer.java:350)
at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.lambda$enrich$5(OssIndexAnalyzer.java:256)
Failed to fetch component-report for: pkg:maven/com.google.guava/[email protected]
java.lang.NullPointerException
at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.transform(OssIndexAnalyzer.java:350)
at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.lambda$enrich$5(OssIndexAnalyzer.java:256)
The CVEs are the following:
h2-2.1.212.jar : CVE-2018-14335
ktor-server-core-1.2.6.jar : CVE-2021-25761
ktor-server-netty-1.2.6.jar : CVE-2020-26129
spring-security-crypto-5.7.1.jar : CVE-2020-5408
xercesImpl-2.12.0.jar : CVE-2022-23437
Everything had been working ok before yesterday afternoon.
I would like to work around this temporarily by just turning off the OSS Index analyser entirely.
I thought I would be able to achieve this (Maven plugin here) by:
<plugin> <groupId>org.owasp</groupId> <artifactId>dependency-check-maven</artifactId> <version>7.1.0</version> <configuration> <!-- Turn off the OSS Index analyser. Currently causing problems with format change and rate limiting. See assorted Github issues e.g. https://github.com/jeremylong/DependencyCheck/issues/4527. --> <ossindexAnalyzerEnabled>false</ossindexAnalyzerEnabled> </configuration> </plugin>But this isn't working as I expected. The analyser is still running and throwing out the errors as documented by others in this issue. What am I missing? How do I just entirely disable the OSS Index analyser if not this way?
Using ossindexAnalyzerEnabled works for me. I use a build job, which do something like:
mvn org.owasp:dependency-check-maven:7.1.0:check -DskipProvidedScope=true -DskipRuntimeScope=true -DprettyPrint=true -DfailBuildOnCVSS=7.0 -DsuppressionFiles="..." -DossindexAnalyzerEnabled=false
No fetching issues / warnings at the moment.
Looking at the logs it looks like dependency-check tries to access https://ossindex.sonatype.org/api/v3/component-report Viewing that in a browser gives a 405 error. That may be irrelevant, but the page does have a link on to this one: https://ossindex.sonatype.org/updates-notice which points out several breaking changes.
@davejbur I have tried to perform a post to that link (https://ossindex.sonatype.org/api/v3/component-report) using this body (as highlighted in a comment above)
{
"coordinates": [
"pkg:maven/xerces/[email protected]"
]
}
and I receive a response describing the vulnerability stating that it isn't a CVE.
So I don't know if these are actual CVE vulnerabilities or not... I am asking because it is breaking some builds
@antonilic these are warnings about NullPointerExceptions. They don't break any builds.
Ok thanks. In my case the build gets broken because of vulnerabilities raised by the oss index
Indeed - the build "breaks" in that it can't be completed. As a temporary workaround, I'm using the following configuration:
<plugin>
<groupId>org.owasp</groupId>
<artifactId>dependency-check-maven</artifactId>
<version>7.1.0</version>
<executions>
<execution>
<configuration>
<ossindexAnalyzerEnabled>false</ossindexAnalyzerEnabled>
</configuration>
<goals>
<goal>check</goal>
</goals>
</execution>
</executions>
</plugin>
This compiles, skipping the remote OSS index check. Not great from a security/vulnerability checking point of view (the whole point of doing this in the first place:-)) but at least it gets past this hiccup for now.
In gradle, I have done something like this and it works:
dependencyCheck {
analyzers.ossIndexEnabled = false
}
I am using the plugin 7.1.0.1 for the dependency check.
However I am not sure that disabling this analyser is the right solution. It could be ok ish for now because in my case the vulnerabilities are marked as non CVE.
Hey all Sonatype Product Manager for OSS Index here. Firstly sorry this has caused you all an issue. There are a number of vulnerabilities that have been found by our research teams that haven't made their way to the public databases yet. One of the huge advantages of the upgrade me made yesterday is that anyone using OSSI now benefits from research done by Sonatype's commercial research teams, making this the highest quality free data source.
These new vulnerabilities that don't have CVEs will now have SONATYPE-* IDs which I think might be causing the problem. For anonymous access we've bundled any of these new vulnerabilities up and only display the ID of the vulnerability with the highest severity. To expand them out you'll need to be authenticated.
We made a temporary fix last night to remove the aggregated vulnerability from anonymous access whenever we see a dependency-track user-agent hoping that would fix this problem. I'm guessing some of you folks are authenticated and are still running into trouble? That's probably going to require a change to dependency-track to support the new ID format.
@jeremylong
- We can filter the new vulnerabilities out temporarily, that's not idea but will allow Dependency-track to keep working.
- I'll also have the Sonatype Lift / OSSI team take a look at whether we can contribute a fix to Dependency-track
I'll reply here when I have more information.
@jlstephens89 have a look at the other recent issues. Many users are now being rate-limited when they weren't before, and OSS Index has added various (apparently) new errors causing false positives.
A high-quality free public data source that wanted to improve everyone's software would not require a login in order to see it.
@OrangeDog On it. https://github.com/jeremylong/DependencyCheck/issues/4535#issuecomment-1137191625
@OrangeDog https://github.com/jeremylong/DependencyCheck/issues/4535#issuecomment-1137233391
Similar error, I think on an older version.
[error] org.owasp.dependencycheck.analyzer.exception.AnalysisException: Failed to request component-reports
[error] at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.analyzeDependency(OssIndexAnalyzer.java:149)
[error] at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze(AbstractAnalyzer.java:131)
[error] at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:88)
[error] at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:37)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] at java.lang.Thread.run(Thread.java:748)
[error] Caused by: java.lang.NullPointerException
[error] at org.sonatype.ossindex.service.client.cache.DirectoryCache.entryKey(DirectoryCache.java:149)
[error] at org.sonatype.ossindex.service.client.cache.DirectoryCache.entryFile(DirectoryCache.java:157)
[error] at org.sonatype.ossindex.service.client.cache.DirectoryCache.putAll(DirectoryCache.java:134)
[error] at org.sonatype.ossindex.service.client.internal.OssindexClientImpl.requestComponentReports(OssindexClientImpl.java:171)
[error] at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.requestReports(OssIndexAnalyzer.java:197)
[error] at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.analyzeDependency(OssIndexAnalyzer.java:138)
[error] ... 7 more
Similar error, I think on an older version.
[error] org.owasp.dependencycheck.analyzer.exception.AnalysisException: Failed to request component-reports [error] at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.analyzeDependency(OssIndexAnalyzer.java:149) [error] at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze(AbstractAnalyzer.java:131) [error] at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:88) [error] at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:37) [error] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [error] at java.lang.Thread.run(Thread.java:748) [error] Caused by: java.lang.NullPointerException [error] at org.sonatype.ossindex.service.client.cache.DirectoryCache.entryKey(DirectoryCache.java:149) [error] at org.sonatype.ossindex.service.client.cache.DirectoryCache.entryFile(DirectoryCache.java:157) [error] at org.sonatype.ossindex.service.client.cache.DirectoryCache.putAll(DirectoryCache.java:134) [error] at org.sonatype.ossindex.service.client.internal.OssindexClientImpl.requestComponentReports(OssindexClientImpl.java:171) [error] at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.requestReports(OssIndexAnalyzer.java:197) [error] at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.analyzeDependency(OssIndexAnalyzer.java:138) [error] ... 7 more
Hmm from that stack trace it looks like the the null coordinates has been cached in some way. Looking at the OSSI client code this is getting cached in your user directory. We've just released more bug fixes which should help so try again and if that still fails, try clearing the cache.
Same problem here but not with maven, is with flask
[WARN] Failed to fetch component-report for: pkg:pypi/[email protected] java.lang.NullPointerException: null at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.transform(OssIndexAnalyzer.java:325) at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.lambda$enrich$5(OssIndexAnalyzer.java:231) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:4[72](https://gitlab.falabella.com/falabella-retail/txd/IA/automatizaciones/bot-mkp/mkp-get-business-report/-/jobs/6866345#L72)) at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:1[73](https://gitlab.falabella.com/falabella-retail/txd/IA/automatizaciones/bot-mkp/mkp-get-business-report/-/jobs/6866345#L73)) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.forEachOrdered(ReferencePipeline.java:490) at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.enrich(OssIndexAnalyzer.java:232) at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.analyzeDependency(OssIndexAnalyzer.java:156) at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze(AbstractAnalyzer.java:131) at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:88) at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:37) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:[74](https://gitlab.falabella.com/falabella-retail/txd/IA/automatizaciones/bot-mkp/mkp-get-business-report/-/jobs/6866345#L74)8)
Currently, I keep getting NPEs for these libraries (dependency-check-maven plugin 7.1.0):
pkg:maven/com.fasterxml.jackson.core/[email protected]
pkg:maven/org.apache.commons/[email protected]
pkg:maven/org.bouncycastle/[email protected]
pkg:maven/com.squareup.okhttp/[email protected]
pkg:maven/org.apache.xmlgraphics/[email protected]
pkg:maven/com.fasterxml.woodstox/[email protected]
pkg:maven/com.twelvemonkeys.imageio/[email protected]
pkg:maven/org.terracotta/[email protected]
As suggested by the others, setting <ossindexAnalyzerEnabled>false</ossindexAnalyzerEnabled> in the POM helped me get rid of these NPEs. But of course, I'd rather have my project dependencies checked against OSS Index again...
Addendum 1: Stacktrace is always the same:
[WARNING] Failed to fetch component-report for: pkg:maven/com.fasterxml.jackson.core/[email protected]
java.lang.NullPointerException
at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.transform (OssIndexAnalyzer.java:350)
at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.lambda$enrich$5 (OssIndexAnalyzer.java:256)
at java.util.stream.ReferencePipeline$3$1.accept (ReferencePipeline.java:193)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining (ArrayList.java:1384)
at java.util.stream.AbstractPipeline.copyInto (AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto (AbstractPipeline.java:472)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential (ForEachOps.java:150)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential (ForEachOps.java:173)
at java.util.stream.AbstractPipeline.evaluate (AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEachOrdered (ReferencePipeline.java:490)
at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.enrich (OssIndexAnalyzer.java:257)
at org.owasp.dependencycheck.analyzer.OssIndexAnalyzer.analyzeDependency (OssIndexAnalyzer.java:163)
at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze (AbstractAnalyzer.java:131)
at org.owasp.dependencycheck.AnalysisTask.call (AnalysisTask.java:88)
at org.owasp.dependencycheck.AnalysisTask.call (AnalysisTask.java:37)
at java.util.concurrent.FutureTask.run (FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
at java.lang.Thread.run (Thread.java:748)
Addendum 2: Weirdly enough, the Jenkins plugin doesn't seem to be affected.
@uwesinha have you tried in the last 20 mins? I've passed your message onto the team and we will verify those specific packages.
Thanks for looking into this, @jlstephens89! I ran my build again 8 minutes ago, but sadly no change. 😢 (I'm in CEST land, so I'll check back tomorrow. 😃)
Thank you, i have run the build with gradle and I am not seeing the null pointer exc anymore. I see that there are some vulnerabilities that should be solved our side.
@uwesinha I made a small project with your packages, but was not able to reproduce the exception. I wonder if the data in the dependency-check cache is corrupted. I would suggest deleting the OSS Index portion of the cache and rerunning.
I found it in the <dependency-check install dir>/data/oss_cache folder. Clear that out and see if it resolves the problem.
@uwesinha Another edge case has been resolved which may also have been the cause for your NPE.
Thanks all - I've just retried with ossindexAnalyzerEnabled set to true again, and it all works fine!
We at Sonatype really appreciate everyone's patience as we work through these issues. I can assure you we did extensively test these changes over the last 2 months but there is always room for improvement and the next change we make we'll be doing a lot more testing with the various clients and looking for ways to more gradually roll out. You all reporting and raising the issues has enabled us to react as fast as possible and you've all been a great help. Thanks!
Also if you ever need to reach us directly you can at [email protected]
Thanks for the service in the first place!
As client users we should probably give the maintainer a hand in making the library a bit more resilient to unexpected formatted data so that the tool handles it more gracefully.
I’ll try to find some time to help with that over the coming week.
Thanks erveryone, esp. @ken-duck, for analyzing this problem! After deleting the OSS-Index-related portion of the cache the NPE is gone at last.
BTW: For the Maven plugin (version 7.x) the cache directories can be found under $HOME/.m2/repository/org/owasp/dependency-check-data/7.0.
Alteranively one can issue the command
mvn dependency-check:purge
https://jeremylong.github.io/DependencyCheck/dependency-check-maven/purge-mojo.html
@jeremylong @jlstephens89 Think we can close this now? Or are there still reports in the comments that require follow-up?
@aikebah as far as I understand, dependency-check still needs updating to handle the change. Then OSSIndex can stop hiding it from dependency-check user agents.
@aikebah as far as I understand, dependency-check still needs updating to handle the change. Then OSSIndex can stop hiding it from dependency-check user agents.
Ah right... after carefully reading the history I see that sonatype currently has a quickfix in place to hide the problematic entries for dependencycheck
A local test with a modified current snapshot (modified user-agent, so that useragent based response filtering at Sonatype OSSIndex will not kick in) appears to suggest that the issue is resolved.
I got proper reports for both anonymous access and authenticated access for pkg:maven/io.netty/[email protected] as well as pkg:maven/com.fasterxml.jackson.core/jackson-core
Unauthenticated yields a report with a 1 vulnerability found (OSSINDEX)
Authenticated (after clearing cache) yields the detailed sonatype vulnerability
@jlstephens89 @ken-duck can you confirm that https://github.com/sonatype/ossindex-public/issues/35#issuecomment-1141267867 essentially means that this issue has been resolved by internal fixes in the OSSIndex? Or did I overlook some cases?
The only part I wasn't able to test is a result of multiple sonatype internal vulnerability records for a single component. The packageURLs that I checked from several of the NPE reports all appear to have only a single sonatype internal vulnerability record.
@aikebah It is definitely my understanding that the comment you linked means that the issue has been resolved by internal fixes. The fundamental problem was that the "reference" was not being set in some cases and that wreaked havoc on several downstream tools. The quick fix hid these things, but a proper fix has been applied.
In addition (and off topic), a few recent fixes have also improved the filling in of the "cve" field when a vulnerability has a sonatype-#### Id, but subsequently is assigned a CVE number. In those cases the "id" will remain as a sonatype-#### id, but the cve field will be appropriately filled in.
In either case the reference should be assigned.
If we get enough community feedback that supports no longer filtering the "sonatype-####" from the unauthenticated results we would certainly be willing and happy to do so.
If we get enough community feedback that supports no longer filtering the "sonatype-####" from the unauthenticated results we would certainly be willing and happy to do so.
@ken-duck so does that mean that it's intentional that unauthenticated DependencyCheck requests aren't currently returning all the same vulnerabilities an authenticated or non-DependencyCheck request does?
I recently raised an issue, https://github.com/sonatype/ossindex-public/issues/37, about a discrepancy I've been seeing between what's being returned from OSS based on your user-agent. It's causing my Jenkins build server (which authenticates for better rate limiting) to have a different outcome than my local scans (which doesn't bother authenticating) that's taken a few days to lead me here.
It seems it would be better to have the full list of vulnerabilities returned regardless of authentication now that DependencyCheck has fixed the issue, no? Otherwise there would be what I imagine is a large group of people potentially missing vulnerabilities due to never authenticating as rate limiting isn't a concern for them.
