Nested provenance cannot be resolved because SVN path is not found

[bennati]

When scanning https://github.com/apache/maven-doxia/tree/doxia-1.1.2/doxia-core The scanner returns the following error:

Could not resolve nested provenance for package 'Maven:xml-apis:xml-apis:1.3.03': SVNException: svn: E160005: Target path '/xerces/xml-commons' does not exist svn: E175002: REPORT of '/repos/asf/!svn/vcc/default': 500 Internal Server Error (https://svn.apache.org)

Despite the path existing https://svn.apache.org/repos/asf/xerces/xml-commons/tags/xml-commons-external-1_3_03/

This is our curation, that has worked until now:

- id: "Maven:xml-apis:xml-apis:1.3.03"
  curations:
    ...
    vcs:
      type: "Subversion"
      url: "https://svn.apache.org/repos/asf/xerces/xml-commons"
      revision: "tags/xml-commons-external-1_3_03"

[sschuberth]

500 Internal Server Error (https://svn.apache.org)

This sounds like a temporary issue with the server itself. Can you just try again @bennati?

[bennati]

I tried again and the issue is still present. You can verify it as well: Scanned revision bf37c3f8745e1e68524ff8184c69615bb4562125 of Git repository https://github.com/apache/maven-doxia.git

[bennati]

I debugged the scanner and found that the correct VCS_URL is passed to https://github.com/oss-review-toolkit/ort/blob/35d92523ad2cc9c536662dd1afa0d27c0dd124c8/downloader/src/main/kotlin/vcs/Subversion.kt#L112

The issue seems to be inside org.tmatesoft.svn.core.wc.SVNClientManager, as it tries to split the VCS_URL into a base URL and path, where the base URL is https://svn.apache.org/repos/asf/!svn/vcc/default (which does not work)

Below is the stacktrace:

org.tmatesoft.svn.core.SVNException: svn: E160005: Target path '/xerces/xml-commons' does not exist
svn: E175002: REPORT of '/repos/asf/!svn/vcc/default': 500 Internal Server Error (https://svn.apache.org)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1365)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:859)
	at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:263)
	at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
	at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
	at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
	at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
	at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
	at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
	at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
	at org.ossreviewtoolkit.downloader.vcs.Subversion.updateEmptyPath(Subversion.kt:116)
	at org.ossreviewtoolkit.downloader.vcs.Subversion.updateWorkingTree-BWLJW6A(Subversion.kt:138)
	at org.ossreviewtoolkit.downloader.VersionControlSystem.updateWorkingTree-BWLJW6A$default(VersionControlSystem.kt:361)
	at org.ossreviewtoolkit.scanner.provenance.DefaultNestedProvenanceResolver$resolveNestedRepository$5.invoke(NestedProvenanceResolver.kt:83)
	at org.ossreviewtoolkit.scanner.provenance.DefaultNestedProvenanceResolver$resolveNestedRepository$5.invoke(NestedProvenanceResolver.kt:82)
	at org.ossreviewtoolkit.scanner.utils.DefaultWorkingTreeCache.use(WorkingTreeCache.kt:65)
	at org.ossreviewtoolkit.scanner.provenance.DefaultNestedProvenanceResolver.resolveNestedRepository(NestedProvenanceResolver.kt:82)
	at org.ossreviewtoolkit.scanner.provenance.DefaultNestedProvenanceResolver.access$resolveNestedRepository(NestedProvenanceResolver.kt:47)
	at org.ossreviewtoolkit.scanner.provenance.DefaultNestedProvenanceResolver$resolveNestedProvenance$1.invokeSuspend(NestedProvenanceResolver.kt:56)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:284)
	at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
	at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
	at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
	at org.ossreviewtoolkit.scanner.provenance.DefaultNestedProvenanceResolver.resolveNestedProvenance(NestedProvenanceResolver.kt:56)
	at org.ossreviewtoolkit.scanner.Scanner$resolveNestedProvenances$duration$1$1$1$1.invokeSuspend(Scanner.kt:207)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
14:03:49.136 [DefaultDispatcher-worker-2] WARN  org.ossreviewtoolkit.downloader.vcs.Subversion - Failed to update the Subversion working tree at '/tmp/ort-DefaultWorkingTreeCache3204553652592340692' to revision '380970':
SVNException: svn: E160005: Target path '/xerces/xml-commons' does not exist
svn: E175002: REPORT of '/repos/asf/!svn/vcc/default': 500 Internal Server Error (https://svn.apache.org)

[bennati]

Any opinion on whether this is a bug in the SW or in the Apache SVN server? Any suggestions on how to fix this?

[bennati]

any updates on this? @sschuberth

[sschuberth]

No, I didn't yet find the time to look into this.

[bennati]

any updated?

[sschuberth]

I finally invested some of my personal time to look into this, and I was able to come up with a test to reproduce this. It's not related to the downloader, but to the (nested) provenance resolver. Looks like we're having a SVN-specific issue with deciding whether to update the working tree with a (numeric) revision, or a switch to a (symbolic) tag.

[sschuberth]

@bennati, could you please try whether https://github.com/oss-review-toolkit/ort/pull/8572 fixes the issue for you? (Note, cloning that repo might actually take a long time...)