Scan step crashes with SVNException

[schvvarzekatze]

When using

For Gradle packages the scan crashes with an error

The same config including environment is only working and generation successful reports for Yarn as package manager.

Question: Are there special config changes to be considered to make the recent ORT version run as well for Gradle in the scan step for libraries from Subversion as verson control system type? Does it help to exclude packages like this or could exceptions be caught?

To Reproduce

Steps to reproduce the behavior:

1. Create flattened project with all dependencies
2. Start ORT run with analyzer and scan for Package Manager Gradle

Expected behavior

All relevant libraries determined from the analyzer result are scanned

Console / log output

Logs

03:07:38.539 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Scanning RepositoryProvenance(vcsInfo=VcsInfo(type=Subversion, url=http://svn.apache.org/repos/asf/xml/commons, revision=tags/xml-commons-resolver-1_2, path=), resolvedRevision=1905005) (89 of 387)...
Exception in thread "main" java.io.IOException: Unable to initialize a Subversion working tree in '/tmp/ort-DefaultWorkingTreeCache10536384807322232179' from http://svn.apache.org/repos/asf/xml/commons.
	at org.ossreviewtoolkit.plugins.versioncontrolsystems.subversion.Subversion.initWorkingTree(Subversion.kt:100)
	at org.ossreviewtoolkit.downloader.DefaultWorkingTreeCache.getWorkingTree(WorkingTreeCache.kt:85)
	at org.ossreviewtoolkit.downloader.DefaultWorkingTreeCache.use(WorkingTreeCache.kt:65)
	at org.ossreviewtoolkit.scanner.provenance.DefaultProvenanceDownloader.downloadFromVcs(ProvenanceDownloader.kt:104)
	at org.ossreviewtoolkit.scanner.provenance.DefaultProvenanceDownloader.access$downloadFromVcs(ProvenanceDownloader.kt:80)
	at org.ossreviewtoolkit.scanner.provenance.DefaultProvenanceDownloader$download$1.invokeSuspend(ProvenanceDownloader.kt:96)
	at org.ossreviewtoolkit.scanner.provenance.DefaultProvenanceDownloader$download$1.invoke(ProvenanceDownloader.kt)
	at org.ossreviewtoolkit.scanner.provenance.DefaultProvenanceDownloader$download$1.invoke(ProvenanceDownloader.kt)
	at org.ossreviewtoolkit.utils.ort.UtilsKt$runBlocking$1.invokeSuspend(Utils.kt:231)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:101)
	at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:263)
	at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:95)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:69)
	at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
	at org.ossreviewtoolkit.utils.ort.UtilsKt.runBlocking(Utils.kt:231)
	at org.ossreviewtoolkit.utils.ort.UtilsKt.runBlocking$default(Utils.kt:229)
	at org.ossreviewtoolkit.scanner.provenance.DefaultProvenanceDownloader.download(ProvenanceDownloader.kt:96)
	at org.ossreviewtoolkit.scanner.Scanner.scanPath(Scanner.kt:568)
	at org.ossreviewtoolkit.scanner.Scanner.runPathScanners(Scanner.kt:445)
	at org.ossreviewtoolkit.scanner.Scanner.scan(Scanner.kt:177)
	at org.ossreviewtoolkit.scanner.Scanner$scan$3.invokeSuspend(Scanner.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:101)
	at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:263)
	at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:95)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:69)
	at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:47)
	at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
	at org.ossreviewtoolkit.plugins.commands.scanner.ScannerCommand.runScanners(ScannerCommand.kt:236)
	at org.ossreviewtoolkit.plugins.commands.scanner.ScannerCommand.run(ScannerCommand.kt:142)
	at com.github.ajalt.clikt.core.CoreCliktCommandKt.parse(CoreCliktCommand.kt:107)
	at com.github.ajalt.clikt.core.CoreCliktCommandKt.main(CoreCliktCommand.kt:78)
	at com.github.ajalt.clikt.core.CoreCliktCommandKt.main(CoreCliktCommand.kt:90)
	at org.ossreviewtoolkit.cli.OrtMainKt.main(OrtMain.kt:85)
Caused by: org.tmatesoft.svn.core.SVNException: svn: E170000: URL 'http://svn.apache.org/repos/asf/xml/commons' doesn't exist
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
	at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.checkout(SvnNgAbstractUpdate.java:852)
	at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgCheckout.run(SvnNgCheckout.java:26)
	at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgCheckout.run(SvnNgCheckout.java:11)
	at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
	at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
	at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
	at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doCheckout(SVNUpdateClient.java:777)
	at org.ossreviewtoolkit.plugins.versioncontrolsystems.subversion.Subversion.initWorkingTree(Subversion.kt:89)
	... 35 more

Environment

• ORT version: 35.0.0
• Java version: JDK 21.0.4+7-LTS
• OS: Linux
• ScanCode version: 32.2.1
• 8 CPUs and a maximum of 19664 MiB of memo

And specify (relevant parts of) your ORT configuration (config.yml):

ort:
  forceOverwrite: false
  addAuthorsToCopyrights: true
  enableRepositoryPackageCurations: true 
  enableRepositoryPackageConfigurations: true

  licenseFilePatterns:
    licenseFilenames: [ 'license*' ]
    patentFilenames: [ patents ]
    rootLicenseFilenames: [ 'readme*' ]

  severeIssueThreshold: ERROR
  severeRuleViolationThreshold: ERROR

  analyzer:
    allowDynamicVersions: true
    enabledPackageManagers: [ GradleInspector, Yarn, NPM ]

  downloader:
    allowMovingRevisions: true
    # Only used if the '--license-classifications-file' option is specified.
    includedLicenseCategories:
      - copyleft
      - copyleft-provide-sourcecode
      - weak-copyleft
      - weak-copyleft-provide-sourcecode
      - proprietary
      - permissive
      - public-domain
      - no-assertion
      - not-for-commercial-use
      - include-in-notice-file
      - include-source-code-offer-in-notice-file

  scanner:
    skipConcluded: true

    archive:

      postgresStorage:
        connection:
          url: ${POSTGRES_URL}
          schema: public
          username: ort
          password: ${POSTGRES_PASSWORD}
          sslmode: disable

    createMissingArchives: true

    storages:
      clearlyDefined:
        serverUrl: 'https://api.clearlydefined.io'

      postgres:
        connection:
          url: ${POSTGRES_URL}
          schema: public
          username: ort
          password: ${POSTGRES_PASSWORD}
          sslmode: disable

    storageReaders: [ postgres, clearlyDefined ]
    storageWriters: [ postgres ]

    provenanceStorage:

      postgresStorage:
        connection:
          url: ${POSTGRES_URL}
          schema: public
          username: ort
          password: ${POSTGRES_PASSWORD}
          sslmode: disable

Additional context

cf. Description above

Thank you for more information.

[sschuberth]

This probably has the same root cause as https://github.com/oss-review-toolkit/ort/issues/5232 and is not related to Gradle specifically.

[schvvarzekatze]

With the workaround mentioned https://github.com/oss-review-toolkit/ort/issues/5232 not a relevant bug anymore. Thank you!