SpdxDocumentFile: downloadLocation is not used if the dependency is local

[bennati]

If the dependency is stored in a subfolder, the analyzer infers the VCS URL from the project instead of from the downloadLocation attribute of the spdx file.

This is caused by the VCS info being determined from the working tree first, and from the download location as fallback. https://github.com/oss-review-toolkit/ort/blob/main/analyzer/src/main/kotlin/managers/SpdxDocumentFile.kt#L282

The current implementation makes it impossible to reference a package where the source code is located in another repository than the SPDX file. A scenario where this is no issue is if you put the SPDX file in a fork of the packages source code.

[sschuberth]

This PR probably is related https://github.com/oss-review-toolkit/ort/pull/4277.

[sschuberth]

Maybe also semi-related: https://github.com/oss-review-toolkit/ort/pull/4606.

[tsteenbe]

Below I wrote out current and desired handling of package.spdx.yml by ORT:

Current

@bennati Did a scan of github.com/movetk/movetk/ and master/third_party/GsTL/package.spdx.yml currently shows up as a project. Which is correct as packageFileName: "./" is set indicating that the sources for SPDXRef-Package-GsTL are in the same directory as package.spdx.yml

SPDXID: "SPDXRef-DOCUMENT"
spdxVersion: "SPDX-2.2"
creationInfo:
  created: "2020-09-07T13:30:00Z"
  creators:
  - "Organization: HERE Europe B.V."
  - "Person: Thomas Steenbergen"
  licenseListVersion: "3.9"
name: "GsTL-1.2.1"
dataLicense: "CC0-1.0"
documentNamespace: "http://spdx.org/spdxdocs/spdx-document-GsTL"
documentDescribes:
- "SPDXRef-Package-GsTL"
packages:
- SPDXID: "SPDXRef-Package-GsTL"
  copyrightText: "NOASSERTION"
  description: "GsTL is a C++ library that provides a comprehensive set of tools and algorithms for geostatistics."
  downloadLocation: "http://prdownloads.sourceforge.net/gstl/GsTL-1.2.1.tgz?download"
  filesAnalyzed: false
  homepage: "http://gstl.sourceforge.net/"
  licenseConcluded: "NOASSERTION"
  licenseDeclared: "BSD-4-Clause"
  name: "GsTL"
  sourceInfo: ""
  versionInfo: "1.2.1"
  packageFileName: "./"

Desired

Use Case	ORT check	Package or Project?	VCS?
Use package.spdx.yml to describe a single package that ORT can not detect and the sources of the package are in the same code repository.	packageFileName set to the local sources directory, only one package defined and documentDescribes points to this package.	Project	Code repository containing package.spdx.yml
Use package.spdx.yml to describe a single package that ORT can not detect and the package is downloaded by the build tool during compilation or runtime	packageFileName is omitted, only one package defined and documentDescribes points to this package.	Package, dependency of root project or if not exist create unmanaged	downloadLocation as defined in package.spdx.yml

Note as per the specification in https://github.com/spdx/spdx-spec/issues/439 package.spdx.yml can only be used to describe a single package. If you want to describe multiple you have to use a project.spdx.yml.

[tsteenbe]

@bennati Just told me that when he flipped the logic to use downloadLocation before VCS his scan did not crash .. recommend Bosch to test this as well as they have repositories with a project.spdx.yml pointing to multiple package.spdx.yml files. Thought for a project the source code always had to be local..