sbt-dependency-check icon indicating copy to clipboard operation
sbt-dependency-check copied to clipboard

Issue with downloading CVE file via NVD url.

Open devbyteops opened this issue 11 months ago • 0 comments

I run sbt dependencyCheck for my project on Bitbucket pipeline. Sometimes it fails with an error in downloading gz file from https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.json.gz the error -

06:52:35.960 [pool-7-thread-1] ERROR org.owasp.dependencycheck.utils.HttpResourceConnection - Error retrieving https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.json.gz; received response code 401; Unauthorized
06:52:39.992 [pool-7-thread-1] ERROR org.owasp.dependencycheck.utils.HttpResourceConnection - Error retrieving https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.json.gz; received response code 401; Unauthorized
06:52:44.017 [pool-7-thread-1] ERROR org.owasp.dependencycheck.utils.HttpResourceConnection - Error retrieving https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.json.gz; received response code 401; Unauthorized
06:52:52.041 [pool-7-thread-1] ERROR org.owasp.dependencycheck.utils.HttpResourceConnection - Error retrieving https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.json.gz; received response code 401; Unauthorized
06:52:52.045 [pool-7-thread-1] ERROR org.owasp.dependencycheck.data.update.nvd.DownloadTask - Download Failed for NVD CVE - Modified
Some CVEs may not be reported. Reason: Download failed, unable to copy 'https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.json.gz' to '/tmp/dctemp6c83649b-edf1-4b96-8a0c-5fa6b5a94573/cveModified_264360719180345753.json.gz'; Error downloading file https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.json.gz; unable to connect.
06:52:52.045 [pool-7-thread-1] ERROR org.owasp.dependencycheck.data.update.nvd.DownloadTask - If you are behind a proxy you may need to configure dependency-check to use the proxy.
06:53:00.046 [pool-7-thread-1] ERROR org.owasp.dependencycheck.data.update.nvd.DownloadTask - Error downloading NVD CVE - Modified Reason: Unable to download NVD CVE Modified
06:53:00.047 [pool-5-thread-1] ERROR org.owasp.dependencycheck.Engine - The execution of the download was interrupted
org.owasp.dependencycheck.data.update.exception.UpdateException: The execution of the download was interrupted
	at org.owasp.dependencycheck.data.update.NvdCveUpdater.performUpdate(NvdCveUpdater.java:334) ~[?:?]
	at org.owasp.dependencycheck.data.update.NvdCveUpdater.update(NvdCveUpdater.java:136) ~[?:?]
	at org.owasp.dependencycheck.Engine.doUpdates(Engine.java:900) ~[?:?]
	at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase(Engine.java:705) ~[?:?]
	at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:631) ~[?:?]
	at net.vonbuchholtz.sbt.dependencycheck.DependencyCheckPlugin$.createReport(DependencyCheckPlugin.scala:624) ~[?:?]
	at net.vonbuchholtz.sbt.dependencycheck.DependencyCheckPlugin$.$anonfun$checkTask$6(DependencyCheckPlugin.scala:378) ~[?:?]
	at net.vonbuchholtz.sbt.dependencycheck.DependencyCheckPlugin$.withEngine(DependencyCheckPlugin.scala:647) ~[?:?]
	at net.vonbuchholtz.sbt.dependencycheck.DependencyCheckPlugin$.$anonfun$checkTask$2(DependencyCheckPlugin.scala:376) ~[?:?]
	at net.vonbuchholtz.sbt.dependencycheck.DependencyCheckPlugin$.$anonfun$checkTask$2$adapted(DependencyCheckPlugin.scala:339) ~[?:?]
	at scala.Function1.$anonfun$compose$1(Function1.scala:49) ~[scala-library.jar:?]
	at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:63) ~[collections_2.12-1.10.5.jar:1.10.5]
	at sbt.std.Transform$$anon$4.work(Transform.scala:69) ~[task-system_2.12-1.10.5.jar:1.10.5]
	at sbt.Execute.$anonfun$submit$2(Execute.scala:283) ~[tasks_2.12-1.10.5.jar:1.10.5]
	at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:24) ~[util-control_2.12-1.10.5.jar:1.10.5]
	at sbt.Execute.work(Execute.scala:292) ~[tasks_2.12-1.10.5.jar:1.10.5]
	at sbt.Execute.$anonfun$submit$1(Execute.scala:283) ~[tasks_2.12-1.10.5.jar:1.10.5]
	at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265) ~[tasks_2.12-1.10.5.jar:1.10.5]
	at sbt.CompletionService$$anon$2.call(CompletionService.scala:65) ~[tasks_2.12-1.10.5.jar:1.10.5]
	at java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
	at java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: java.util.concurrent.ExecutionException: org.owasp.dependencycheck.utils.DownloadFailedException: Unable to download NVD CVE Modified
	at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:?]
	at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:?]
	at org.owasp.dependencycheck.data.update.NvdCveUpdater.performUpdate(NvdCveUpdater.java:321) ~[?:?]
	... 24 more
Caused by: org.owasp.dependencycheck.utils.DownloadFailedException: Unable to download NVD CVE Modified
	at org.owasp.dependencycheck.data.update.nvd.DownloadTask.call(DownloadTask.java:145) ~[?:?]
	at org.owasp.dependencycheck.data.update.nvd.DownloadTask.call(DownloadTask.java:44) ~[?:?]

I wanna ask if we could have this dependencyCheckCveUrlModified setting, to accept multiple URLs like a main and some mirror ones if one url fails? Or Some caching option would be much helpful

devbyteops avatar Nov 29 '24 11:11 devbyteops