Trouble running the analyzer / scanner on a directory not under Version Control

[OctagonHex]

Hello, I sucessfully created a scan result with the following command. (I used the scancode-toolkit examples.) cli\build\install\ort\bin\ort scan -p "C:\scancode-toolkit-3.1.1\samples" -o myOut Output:

Using scanner 'ScanCode' with storage 'FileBasedStorage with XZCompressedLocalFileStorage backend'.
Local file storage has 0 scan results files.
Writing scan result to 'myOut\scan-result.yml'.

If I look at the .yml file, it looks good and contains many licensed.

Now I try to generate any kind of report. My goal is to generate an attribution notice. So I run the command: cli\build\install\ort\bin\ort report -i myOut\scan-result.yml -o myOutReport -f Excel but the output shows the error

Creating the 'Excel' report...
15:15:23.583 [main] ERROR org.ossreviewtoolkit.commands.ReporterCommand - Could not create 'Excel' report: IllegalArgumentException: The provided ORT result does not contain an analyzer result.
Failed to create any report.

For -f NoticeSummary, or -f NoticeByPackage OSS-RT seems to work at first glance:

Creating the 'NoticeSummary' report...
Successfully created the 'NoticeSummary' report at [myOutReport\NOTICE_SUMMARY] in 0.012422699s.
Successfully created 1 of 1 report(s).

But despite the many licenses in the .yml, the resulting report is empty, i.e. it says: This project neither contains or depends on any third-party software components.

What is the problem, or how can this be fixed?

I attached my scan result for easy reference. scan-result.yml.txt

[sschuberth]

While running a reporter on an ORT result with only a scan result is not forbidden, this is a use-case that is not well tested. The usual (and well tested) workflow is to first create an ORT result with an analyzer result, and then use that as the input for the scanner, which creates another ORT result file that combines the analyzer and scan results. Such "rich" ORT result files should work fine to create reports.

[OctagonHex]

Maybe you can give me a hint on how to accomplish my goal. For example: I try to anaylze a unstructured directory of source code. I'll use the samples from ScanCode-toolkit. I first analyze them, and the analyzer runs OK. As expected, the result is very short and does not contain any dependencies. Now, the problem is, that If I use this as input, the scanner does not even scan the directory! The output from the anaylzer does not even cotain the source directory. The scanner result now mostly contains "No source artifact URL provided for 'Unmanaged::ScanCode-Samples:'." I also tried to add the project to a local GIT repository (without a remote master), so now the warning for "non-cacheable results" is gone, but the scanner still can't find the source code.

What parameters need to be set, so the analyzer will save where the source code was, so that the scanner can find it?

C:\oss-review-toolkit>cli\build\install\ort\bin\ort --info analyze -f JSON -i "C:\temp\ScanCode-Samples" -o analyzerOut
________ _____________________
\_____  \\______   \__    ___/ the OSS Review Toolkit, version 0.1.0-SNAPSHOT.
 /   |   \|       _/ |    |    Running 'analyze' under Java 14.0.1 on Windows 10 with
/    |    \    |   \ |    |    ORT_DATA_DIR = C:\Users\USER\.ort
\_______  /____|_  / |____|    OS = Windows_NT
        \/       \/
More environment variables:
COMSPEC = C:\WINDOWS\system32\cmd.exe
JAVA_HOME = C:\jdk-14.0.1+7

The following package managers are activated:
        Bower, Bundler, Cargo, Conan, DotNet, GoDep, GoMod, Gradle, Maven, NPM, NuGet, PhpComposer, PIP, Pipenv, Pub, SBT, Stack, Yarn
Analyzing project path:
        C:\temp\ScanCode-Samples
08:16:19.253 [main] INFO  org.ossreviewtoolkit.analyzer.Analyzer - Unmanaged projects found in:
08:16:19.255 [main] INFO  org.ossreviewtoolkit.analyzer.Analyzer -      .
08:16:19.298 [Analyzer-1] INFO  org.ossreviewtoolkit.analyzer.PackageManager - Resolving Unmanaged dependencies for 'C:\temp\ScanCode-Samples'...
08:16:19.358 [Analyzer-1] INFO  org.ossreviewtoolkit.utils.OrtAuthenticator - Authenticator is already installed.
08:16:19.359 [Analyzer-1] INFO  org.ossreviewtoolkit.utils.OrtProxySelector - Proxy selector is already installed.
08:16:19.490 [Analyzer-1] INFO  org.ossreviewtoolkit.utils.OrtAuthenticator - Authenticator is already installed.
08:16:19.491 [Analyzer-1] INFO  org.ossreviewtoolkit.utils.OrtProxySelector - Proxy selector is already installed.
08:16:20.440 [Analyzer-1] WARN  org.ossreviewtoolkit.analyzer.managers.Unmanaged - Analysis of local directory 'C:\temp\ScanCode-Samples' which is not under version control will produce non-cacheable results as no version for the cache key can be determined.
08:16:20.445 [Analyzer-1] INFO  org.ossreviewtoolkit.analyzer.PackageManager - Resolving Unmanaged dependencies for 'ScanCode-Samples' took 1.1431624s.
Found 1 project(s) in total.
Writing analyzer result to 'analyzerOut\analyzer-result.json'.

C:\oss-review-toolkit>cli\build\install\ort\bin\ort --info scan -i analyzerOut\analyzer-result.json -o myOut
________ _____________________
\_____  \\______   \__    ___/ the OSS Review Toolkit, version 0.1.0-SNAPSHOT.
 /   |   \|       _/ |    |    Running 'scan' under Java 14.0.1 on Windows 10 with
/    |    \    |   \ |    |    ORT_DATA_DIR = C:\Users\USER\.ort
\_______  /____|_  / |____|    OS = Windows_NT
        \/       \/
More environment variables:
COMSPEC = C:\WINDOWS\system32\cmd.exe
JAVA_HOME = C:\jdk-14.0.1+7

Using scanner 'ScanCode' with storage 'FileBasedStorage with XZCompressedLocalFileStorage backend'.
Local file storage has 0 scan results files.
08:21:00.843 [main] INFO  org.ossreviewtoolkit.scanner.LocalScanner - Bootstrapping scanner 'ScanCode' as required version 3.0.2 was not found in PATH.
08:21:00.846 [main] INFO  org.ossreviewtoolkit.scanner.scanners.ScanCode - Downloading ScanCode from https://github.com/nexB/scancode-toolkit/archive/v3.0.2.zip...
08:21:02.056 [main] INFO  org.ossreviewtoolkit.scanner.scanners.ScanCode - Retrieved ScanCode from local cache.
08:21:02.497 [main] INFO  org.ossreviewtoolkit.scanner.scanners.ScanCode - Unpacking 'C:\Users\USER\AppData\Local\Temp\ort9967510014256878867ScanCode-v3.0.2.zip' to 'C:\Users\USER\AppData\Local\Temp\ort15900786484210730018ScanCode-3.0.2'...
08:21:49.381 [main] INFO  org.ossreviewtoolkit.utils.ProcessCapture - Running 'C:\Users\USER\AppData\Local\Temp\ort15900786484210730018ScanCode-3.0.2\scancode-toolkit-3.0.2\scancode.bat --version' in 'C:\Users\USER\AppData\Local\Temp\ort15900786484210730018ScanCode-3.0.2\scancode-toolkit-3.0.2'...
08:22:47.472 [main] INFO  org.ossreviewtoolkit.utils.ProcessCapture - Running 'C:\Users\USER\AppData\Local\Temp\ort15900786484210730018ScanCode-3.0.2\scancode-toolkit-3.0.2\scancode.bat --version' in 'C:\Users\USER\AppData\Local\Temp\ort15900786484210730018ScanCode-3.0.2\scancode-toolkit-3.0.2'...
08:22:49.353 [FileBasedStorage with XZCompressedLocalFileStorage backend-1] INFO  kotlinx.coroutines.CoroutineScope - Looking for stored scan results for Unmanaged::ScanCode-Samples: and ScannerDetails(name=ScanCode, version=3.0.2, configuration=--copyright --license --ignore *.ort.yml --info --strip-root --timeout 300 --ignore HERE_NOTICE --ignore META-INF/DEPENDENCIES --json-pp) (1/1).
08:22:49.370 [ScanCode-1] INFO  kotlinx.coroutines.CoroutineScope - No stored result found for Unmanaged::ScanCode-Samples: and ScannerDetails(name=ScanCode, version=3.0.2, configuration=--copyright --license --ignore *.ort.yml --info --strip-root --timeout 300 --ignore HERE_NOTICE --ignore META-INF/DEPENDENCIES --json-pp), scanning package in thread 'ScanCode-1' (1/1).
08:22:49.373 [ScanCode-1] INFO  org.ossreviewtoolkit.downloader.Downloader - Trying to download source code for 'Unmanaged::ScanCode-Samples:'.
08:22:49.377 [ScanCode-1] INFO  org.ossreviewtoolkit.downloader.Downloader - Trying to download 'Unmanaged::ScanCode-Samples:' sources to 'C:\oss-review-toolkit\myOut\downloads\Unmanaged\unknown\ScanCode-Samples\unknown' from VCS...
08:22:49.380 [ScanCode-1] INFO  org.ossreviewtoolkit.downloader.Downloader - Trying to download source artifact for 'Unmanaged::ScanCode-Samples:' from ...
08:22:49.384 [ScanCode-1] ERROR org.ossreviewtoolkit.scanner.LocalScanner - Could not download 'Unmanaged::ScanCode-Samples:': DownloadException: Download failed for 'Unmanaged::ScanCode-Samples:'.
Suppressed: DownloadException: No VCS URL provided for 'Unmanaged::ScanCode-Samples:'.,
Suppressed: DownloadException: No source artifact URL provided for 'Unmanaged::ScanCode-Samples:'.
08:22:49.385 [ScanCode-1] INFO  kotlinx.coroutines.CoroutineScope - Finished scanning Unmanaged::ScanCode-Samples: in thread 'ScanCode-1' (1/1).
08:22:49.388 [main] INFO  org.ossreviewtoolkit.model.OrtResult - Computing excluded projects which may take a while...
08:22:49.390 [main] INFO  org.ossreviewtoolkit.model.OrtResult - Computing excluded projects done.
Writing scan result to 'myOut\scan-result.yml'.

[sschuberth]

I'm trying to sum up the current status here: An OrtResult contains a Repository that in turn contains a VcsInfo. The latter cannot be set to anything meaningful if the analyzed directory is not under version control.

Instead of doing something hacky like setting it to VcsInfo.EMPTY, an idea is to replace the current Repository with something like a new AnalyzerInput class with a Provenance instead of strictly VCS-related classes. Maybe also NestedProvenance could be generalized a bit so AnalyzerInput could use it to also substitute Repository's nestedRepositories. When a directory that is not under version control is analyzed, the provenance would be set to UnknownProvenance.

In that context maybe also RepositoryConfiguration couold be renamed to something more general like ProductConfiguration or so.

[heliocastro]

Easy reproducible (need have git) as simulate a fake monorepo:

mkdir test
cd test 
git clone https://github.com/apple/swift-nio.git
git clone https://github.com/sw360/sw360python.git
ort analyze -i . -o output

[pepper-jk]

I'm having a look at this refactoring. Let me know if you have any more input.