ort icon indicating copy to clipboard operation
ort copied to clipboard

Published 20 hours ago • verified publisher icon oss-review-toolkit

Scanner IOException: Could not delete file '/tmp/ort-DefaultWorkingTreeCache16779531845304394453/test/fixtures/testdir5'

Open schvvarzekatze opened this issue 1 year ago • 8 comments

Describe the bug

The scan step exits with error after file archives for all packages are created.

To Reproduce

Steps to reproduce the behavior:

  1. Build docker image with SSH keys
  2. Run all steps of ORT in gitlab

Expected behavior

All steps are processed sucessfully and reports generated.

Console / log output

Add console and / or log output that shows the error and additional context. No screenshots of plain text please, to keep text searchable.

06:38:08.693 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/node-modules/detect-port.git.
06:38:08.693 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision '5e8f3ed850b09175d76303ee94c8d9276196c45b' with depth limited to 50.
06:38:09.151 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout 5e8f3ed850b09175d76303ee94c8d9276196c45b' in '/tmp/ort-DefaultWorkingTreeCache8631830167380668856'...
06:38:09.419 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader1522870674314798076'...
06:38:09.421 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader1522870674314798076' in 1.635757ms.
06:38:09.425 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader1522870674314798076' to storage in 4.460771ms.
06:38:09.432 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/sindresorhus/is-stream.git.
06:38:09.432 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision 'fb8caed475b4107cee3c22be3252a904020eb2d4' with depth limited to 50.
06:38:09.833 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout fb8caed475b4107cee3c22be3252a904020eb2d4' in '/tmp/ort-DefaultWorkingTreeCache10731298289894898272'...
06:38:10.105 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader11029124658961225776'...
06:38:10.107 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader11029124658961225776' in 1.338771ms.
06:38:10.121 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader11029124658961225776' to storage in 14.009017ms.
06:38:10.127 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/storybookjs/global.git.
06:38:10.127 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision '5a1c2b0554fb8a6a371031627f60eb4596cbb885' with depth limited to 50.
06:38:10.575 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout 5a1c2b0554fb8a6a371031627f60eb4596cbb885' in '/tmp/ort-DefaultWorkingTreeCache3004496325445573908'...
06:38:10.812 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader13987755821299031604'...
06:38:10.814 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader13987755821299031604' in 1.149156ms.
06:38:10.817 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader13987755821299031604' to storage in 3.296815ms.
06:38:10.823 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/benjamn/recast.git.
06:38:10.823 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision '887f5b724bd71ab01877e9ecc378e6aa94d7908c' with depth limited to 50.
06:38:12.615 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout 887f5b724bd71ab01877e9ecc378e6aa94d7908c' in '/tmp/ort-DefaultWorkingTreeCache12407289329936384198'...
06:38:12.866 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader6463690048797873673'...
06:38:12.868 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader6463690048797873673' in 2.385854ms.
06:38:12.872 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader6463690048797873673' to storage in 3.767125ms.
06:38:12.879 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/jshttp/mime-types.git.
06:38:12.879 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision 'ef932231c20e716ec27ea159c082322c3c485b66' with depth limited to 50.
06:38:14.171 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout ef932231c20e716ec27ea159c082322c3c485b66' in '/tmp/ort-DefaultWorkingTreeCache7916550391198994151'...
06:38:14.452 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader302117139644503154'...
06:38:14.453 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader302117139644503154' in 903.721us.
06:38:14.470 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader302117139644503154' to storage in 17.136767ms.
06:38:14.474 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/npm/node-semver.git.
06:38:14.474 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision 'b717044e57bd132c7e5aa50e9af9a03f10d4655a' with depth limited to 50.
06:38:15.232 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout b717044e57bd132c7e5aa50e9af9a03f10d4655a' in '/tmp/ort-DefaultWorkingTreeCache9943611783329226154'...
06:38:15.518 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader4820215469908603632'...
06:38:15.522 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader4820215469908603632' in 3.471959ms.
06:38:15.587 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader4820215469908603632' to storage in 65.004029ms.
06:38:15.598 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/isaacs/path-scurry.git.
06:38:15.598 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision '766c9b06aacef86b43c9999cabc5eb824b5958d6' with depth limited to 50.
06:38:16.033 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout 766c9b06aacef86b43c9999cabc5eb824b5958d6' in '/tmp/ort-DefaultWorkingTreeCache1838279081870557110'...
06:38:16.290 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader9963628392902948812'...
06:38:16.292 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader9963628392902948812' in 1.653177ms.
06:38:16.309 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader9963628392902948812' to storage in 16.683596ms.
06:38:16.313 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/tj/commander.js.git.
06:38:16.313 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision 'e0e723810357e915210af38ccf5098ffe1fb8e65' with depth limited to 50.
06:38:16.854 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout e0e723810357e915210af38ccf5098ffe1fb8e65' in '/tmp/ort-DefaultWorkingTreeCache13114533609521510971'...
06:38:17.132 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader12298078524412593355'...
06:38:17.136 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader12298078524412593355' in 3.555550ms.
06:38:17.391 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader12298078524412593355' to storage in 255.500641ms.
06:38:17.407 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/alexeyraspopov/picocolors.git.
06:38:17.407 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision '228cea3fa726857785b8c069cccc58e9743cd71d' with depth limited to 50.
06:38:17.846 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout 228cea3fa726857785b8c069cccc58e9743cd71d' in '/tmp/ort-DefaultWorkingTreeCache43228604550622083'...
06:38:18.170 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader6604947480578885775'...
06:38:18.172 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader6604947480578885775' in 1.821551ms.
06:38:18.188 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader6604947480578885775' to storage in 16.024302ms.
06:38:18.196 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/sindresorhus/untildify.git.
06:38:18.196 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision '65f6157c8716a122368c8be05a3ad71279196bf9' with depth limited to 50.
06:38:18.640 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout 65f6157c8716a122368c8be05a3ad71279196bf9' in '/tmp/ort-DefaultWorkingTreeCache17916884581051915091'...
06:38:18.907 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader7822260577997241807'...
06:38:18.909 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader7822260577997241807' in 1.352991ms.
06:38:18.924 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader7822260577997241807' to storage in 15.515060ms.
06:38:18.927 [main] INFO  org.ossreviewtoolkit.downloader.Downloader - Trying to download source artifact from https://registry.npmjs.org/commondir/-/commondir-1.0.1.tgz...
06:38:18.930 [main] INFO  org.ossreviewtoolkit.downloader.Downloader - Successfully unpacked https://registry.npmjs.org/commondir/-/commondir-1.0.1.tgz to '/tmp/ort-DefaultProvenanceDownloader12419833723121055923'...
06:38:18.931 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader12419833723121055923'...
06:38:18.932 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader12419833723121055923' in 539.442us.
06:38:18.944 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader12419833723121055923' to storage in 12.140064ms.
06:38:18.947 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/mafintosh/stream-shift.git.
06:38:18.947 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision 'c189b6185a25517593bce28a9c74c8b87b83a11a' with depth limited to 50.
06:38:19.384 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout c189b6185a25517593bce28a9c74c8b87b83a11a' in '/tmp/ort-DefaultWorkingTreeCache3377082997980485001'...
06:38:19.631 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader10181449668420563436'...
06:38:19.632 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader10181449668420563436' in 645.744us.
06:38:19.949 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader10181449668420563436' to storage in 316.835816ms.
06:38:19.956 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/storybookjs/storybook.git.
06:38:19.956 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision '958e86cb02a747793dba3236f44bdc4b31bf9a56' with depth limited to 50.
06:38:20.968 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout 958e86cb02a747793dba3236f44bdc4b31bf9a56' in '/tmp/ort-DefaultWorkingTreeCache16565604389000697255'...
06:38:23.209 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader12757168004669623098'...
06:38:23.307 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader12757168004669623098' in 97.947592ms.
06:38:23.327 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader12757168004669623098' to storage in 19.225614ms.
06:38:23.915 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/mrmlnc/fast-glob.git.
06:38:23.915 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision 'e60a9f5f09bc58a3b22b1d7fb767c25f62df0d07' with depth limited to 50.
06:38:25.668 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout e60a9f5f09bc58a3b22b1d7fb767c25f62df0d07' in '/tmp/ort-DefaultWorkingTreeCache489655750008243161'...
06:38:25.950 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader9211840789053191968'...
06:38:25.953 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader9211840789053191968' in 3.210823ms.
06:38:25.979 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader9211840789053191968' to storage in 25.127698ms.
06:38:25.987 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/unjs/defu.git.
06:38:25.987 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision '1b9fcab2c1479f0295a5f867c6ec36a01fda2dfb' with depth limited to 50.
06:38:26.673 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout 1b9fcab2c1479f0295a5f867c6ec36a01fda2dfb' in '/tmp/ort-DefaultWorkingTreeCache13129054240050079809'...
06:38:26.919 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader4314793008878263429'...
06:38:26.920 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader4314793008878263429' in 919.681us.
06:38:26.935 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader4314793008878263429' to storage in 14.578279ms.
06:38:26.939 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/nodeca/pako.git.
06:38:26.939 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision '85d145e616967f3720382f416da530c23c02ad13' with depth limited to 50.
06:38:28.398 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout 85d145e616967f3720382f416da530c23c02ad13' in '/tmp/ort-DefaultWorkingTreeCache2236626705364083040'...
06:38:28.657 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader7754964271556166836'...
06:38:28.660 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader7754964271556166836' in 1.989775ms.
06:38:28.673 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader7754964271556166836' to storage in 13.727989ms.
06:38:28.681 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/sindresorhus/locate-path.git.
06:38:28.681 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision '3f2dd51787dd7fe22658af7ab817ec542de93b1b' with depth limited to 50.
06:38:29.104 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout 3f2dd51787dd7fe22658af7ab817ec542de93b1b' in '/tmp/ort-DefaultWorkingTreeCache13081117245477695594'...
06:38:29.355 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader4074005149826374330'...
06:38:29.357 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader4074005149826374330' in 1.335550ms.
06:38:29.376 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader4074005149826374330' to storage in 18.687512ms.
06:38:29.382 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/remarkablemark/style-to-object.git.
06:38:29.382 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision 'ccb7575b116f8f8039dea84087aa99891442c233' with depth limited to 50.
06:38:29.797 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout ccb7575b116f8f8039dea84087aa99891442c233' in '/tmp/ort-DefaultWorkingTreeCache1463637688987968845'...
06:38:30.075 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader12614259282808036243'...
06:38:30.077 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader12614259282808036243' in 1.398001ms.
06:38:30.093 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader12614259282808036243' to storage in 15.715635ms.
06:38:30.096 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/visionmedia/debug.git.
06:38:30.096 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision '13abeae468fea297d0dccc50bc55590809241083' with depth limited to 50.
06:38:30.679 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout 13abeae468fea297d0dccc50bc55590809241083' in '/tmp/ort-DefaultWorkingTreeCache13065352815039632379'...
06:38:30.939 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader17932665357109054978'...
06:38:30.942 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader17932665357109054978' in 2.296472ms.
06:38:30.958 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader17932665357109054978' to storage in 16.398620ms.
06:38:30.965 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/ljharb/get-intrinsic.git.
06:38:30.965 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision '3ce1c8e1d677ae3b828f84add6e0e399d3c94a66' with depth limited to 50.
06:38:31.385 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout 3ce1c8e1d677ae3b828f84add6e0e399d3c94a66' in '/tmp/ort-DefaultWorkingTreeCache3535838537575220053'...
06:38:31.651 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader9266256704315120998'...
06:38:31.652 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader9266256704315120998' in 1.413592ms.
06:38:31.657 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader9266256704315120998' to storage in 4.715587ms.
06:38:31.666 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/babel/babel.git.
06:38:31.666 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision '7fd40d86a0d03ff0e9c3ea16b29689945433d4df' with depth limited to 50.
06:38:33.629 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout 7fd40d86a0d03ff0e9c3ea16b29689945433d4df' in '/tmp/ort-DefaultWorkingTreeCache14892066947136215462'...
06:38:38.180 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader14959631236585950034'...
06:38:38.696 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader14959631236585950034' in 515.950519ms.
06:38:38.701 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader14959631236585950034' to storage in 4.223125ms.
06:38:40.103 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/sindresorhus/execa.git.
06:38:40.103 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision 'f4b8b3ab601c94d1503f1010822952758dcc6350' with depth limited to 50.
06:38:40.618 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout f4b8b3ab601c94d1503f1010822952758dcc6350' in '/tmp/ort-DefaultWorkingTreeCache4023791160620417754'...
06:38:40.932 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader12764406246091246381'...
06:38:40.937 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader12764406246091246381' in 4.337178ms.
06:38:40.941 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader12764406246091246381' to storage in 4.526962ms.
06:38:40.954 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/theKashey/aria-hidden.git.
06:38:40.954 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision '52c7374abb9fb0a348c63b9ff613a575868cec62' with depth limited to 50.
06:38:41.375 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout 52c7374abb9fb0a348c63b9ff613a575868cec62' in '/tmp/ort-DefaultWorkingTreeCache10464513147535492578'...
06:38:41.639 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader4522770151403371818'...
06:38:41.640 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader4522770151403371818' in 805.858us.
06:38:41.658 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader4522770151403371818' to storage in 18.140529ms.
06:38:41.663 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from ssh://[email protected]/follow-redirects/follow-redirects.git.
06:38:41.663 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision '35a517c5861d79dc8bff7db8626013d20b711b06' with depth limited to 50.
06:38:44.646 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout 35a517c5861d79dc8bff7db8626013d20b711b06' in '/tmp/ort-DefaultWorkingTreeCache2873071791076153638'...
06:38:45.750 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader12469262497749183941'...
06:38:45.752 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader12469262497749183941' in 1.784230ms.
06:38:45.756 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader12469262497749183941' to storage in 3.840457ms.
06:38:45.764 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/npm/normalize-package-data.git.
06:38:45.764 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision 'dedb606de22fa4bdf21c3679dece5fd07b4a0b1a' with depth limited to 50.
06:38:46.546 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout dedb606de22fa4bdf21c3679dece5fd07b4a0b1a' in '/tmp/ort-DefaultWorkingTreeCache3592582685573421260'...
06:38:46.812 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader18373332533909689987'...
06:38:46.814 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader18373332533909689987' in 2.105228ms.
06:38:46.818 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader18373332533909689987' to storage in 3.381236ms.
06:38:46.824 [main] INFO  org.ossreviewtoolkit.downloader.Downloader - Trying to download source artifact from https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-0.1.7.tgz...
06:38:46.827 [main] INFO  org.ossreviewtoolkit.downloader.Downloader - Successfully unpacked https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-0.1.7.tgz to '/tmp/ort-DefaultProvenanceDownloader2318292349102555522'...
06:38:46.827 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader2318292349102555522'...
06:38:46.828 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader2318292349102555522' in 442.5us.
06:38:46.833 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader2318292349102555522' to storage in 4.669705ms.
06:38:46.836 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/chalk/wrap-ansi.git.
06:38:46.836 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision '115090266b0ebb1797032582de78d617575778ab' with depth limited to 50.
06:38:47.308 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout 115090266b0ebb1797032582de78d617575778ab' in '/tmp/ort-DefaultWorkingTreeCache16083424116224613566'...
06:38:47.599 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader11419342373671712520'...
06:38:47.600 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader11419342373671712520' in 1.326110ms.
06:38:47.604 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader11419342373671712520' to storage in 4.109453ms.
06:38:47.611 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/jonathanong/ee-first.git.
06:38:47.611 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision '512e0ce4cc3643f603708f965a97b61b1a9c0441' with depth limited to 50.
06:38:48.033 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout 512e0ce4cc3643f603708f965a97b61b1a9c0441' in '/tmp/ort-DefaultWorkingTreeCache17942371025228510104'...
06:38:48.289 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader7165531350465037341'...
06:38:48.291 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader7165531350465037341' in 1.183017ms.
06:38:48.311 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader7165531350465037341' to storage in 20.502533ms.
06:38:48.317 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/YuzuJS/setImmediate.git.
06:38:48.317 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision 'f1ccbfdf09cb93aadf77c4aa749ea554503b9234' with depth limited to 50.
06:38:48.783 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout f1ccbfdf09cb93aadf77c4aa749ea554503b9234' in '/tmp/ort-DefaultWorkingTreeCache598796768466539305'...
06:38:49.021 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader13880971386895103527'...
06:38:49.022 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader13880971386895103527' in 759.417us.
06:38:49.030 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader13880971386895103527' to storage in 7.314995ms.
06:38:49.032 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Created file archives for 636 package(s) in 19m 46.436627394s.
06:38:49.899 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'scancode --version' in '/home/ort'...
06:38:50.988 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'scancode --version' in '/home/ort'...
Exception in thread "main" java.io.IOException: Could not delete file '/tmp/ort-DefaultWorkingTreeCache16779531845304394453/test/fixtures/testdir5'.
	at org.ossreviewtoolkit.utils.common.ExtensionsKt.safeDeleteRecursively(Extensions.kt:151)
	at org.ossreviewtoolkit.utils.common.ExtensionsKt.safeDeleteRecursively$default(Extensions.kt:128)
	at org.ossreviewtoolkit.utils.common.ExtensionsKt.safeDeleteRecursively(Extensions.kt:132)
	at org.ossreviewtoolkit.utils.common.ExtensionsKt.safeDeleteRecursively$default(Extensions.kt:128)
	at org.ossreviewtoolkit.utils.common.ExtensionsKt.safeDeleteRecursively(Extensions.kt:132)
	at org.ossreviewtoolkit.utils.common.ExtensionsKt.safeDeleteRecursively$default(Extensions.kt:128)
	at org.ossreviewtoolkit.utils.common.ExtensionsKt.safeDeleteRecursively(Extensions.kt:132)
	at org.ossreviewtoolkit.utils.common.ExtensionsKt.safeDeleteRecursively$default(Extensions.kt:128)
	at org.ossreviewtoolkit.downloader.DefaultWorkingTreeCache.shutdown(WorkingTreeCache.kt:92)
	at org.ossreviewtoolkit.plugins.commands.scanner.ScannerCommand$runScanners$5.invokeSuspend(ScannerCommand.kt:239)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
	at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:277)
	at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:95)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:69)
	at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:48)
	at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
	at org.ossreviewtoolkit.plugins.commands.scanner.ScannerCommand.runScanners(ScannerCommand.kt:239)
	at org.ossreviewtoolkit.plugins.commands.scanner.ScannerCommand.run(ScannerCommand.kt:142)
	at com.github.ajalt.clikt.parsers.Parser.finalizeAndRun(Parser.kt:348)
	at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:218)
	at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:245)
	at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:42)
	at com.github.ajalt.clikt.core.CliktCommand.parse(CliktCommand.kt:457)
	at com.github.ajalt.clikt.core.CliktCommand.parse$default(CliktCommand.kt:454)
	at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:474)
	at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:481)
	at org.ossreviewtoolkit.cli.OrtMainKt.main(OrtMain.kt:85)
section_end:1723531164:step_script
[0Ksection_start:1723531164:upload_artifacts_on_failure
[0K[0K[36;1mUploading artifacts for failed job[0;m[0;m
[32;1mUploading artifacts...[0;m
build/oss-review-toolkit/*: found 4 matching artifact files and directories[0;m 
Uploading artifacts as "archive" to coordinator... 201 Created[0;m  id[0;m=1476438 responseStatus[0;m=201 Created token[0;m=glcbt-64
section_end:1723531166:upload_artifacts_on_failure
[0Ksection_start:1723531166:cleanup_file_variables
[0K[0K[36;1mCleaning up project directory and file based variables[0;m[0;m
section_end:1723531167:cleanup_file_variables
[0K[31;1mERROR: Job failed: exit code 1
[0;m

Environment

Output of the ort requirements command:

<copy & paste console output to here; no screenshots please>

Or manually specify:

  • ORT version: not in the logs probably 27.0
  • Java version: 17
  • OS: run on GitLab CI (Linux) as docker container
  • ScanCode version: 32.2.1
  • 16 CPUs and a maximum of 19664 MiB of memo

And specify (relevant parts of) your ORT configuration (config.yml):

ort:
  forceOverwrite: false
  addAuthorsToCopyrights: true
  enableRepositoryPackageCurations: true 
  enableRepositoryPackageConfigurations: true

  licenseFilePatterns:
    licenseFilenames: [ 'license*' ]
    patentFilenames: [ patents ]
    rootLicenseFilenames: [ 'readme*' ]

  severeIssueThreshold: ERROR
  severeRuleViolationThreshold: ERROR

  analyzer:
    allowDynamicVersions: true
    enabledPackageManagers: [ Gradle, Yarn, NPM ]

  downloader:
    allowMovingRevisions: true
    # Only used if the '--license-classifications-file' option is specified.
    includedLicenseCategories:
      - copyleft
      - copyleft-provide-sourcecode
      - weak-copyleft
      - weak-copyleft-provide-sourcecode
      - proprietary
      - permissive
      - public-domain
      - no-assertion
      - not-for-commercial-use
      - include-in-notice-file
      - include-source-code-offer-in-notice-file

  scanner:
    skipConcluded: true

    archive:

      postgresStorage:
        connection:
          url: ${POSTGRES_URL}
          schema: public
          username: ort
          password: ${POSTGRES_PASSWORD}
          sslmode: disable

    createMissingArchives: true

    storages:
      clearlyDefined:
        serverUrl: 'https://api.clearlydefined.io'

      postgres:
        connection:
          url: ${POSTGRES_URL}
          schema: public
          username: ort
          password: ${POSTGRES_PASSWORD}
          sslmode: disable

    storageReaders: [ postgres, clearlyDefined ]
    storageWriters: [ postgres ]

    provenanceStorage:

      postgresStorage:
        connection:
          url: ${POSTGRES_URL}
          schema: public
          username: ort
          password: ${POSTGRES_PASSWORD}
          sslmode: disable

Additional context

Add any other context about the problem here.

schvvarzekatze avatar Aug 14 '24 06:08 schvvarzekatze

Some questions:

  • Is your GitLab CI running on Linux or on Windows?
  • Can you list more log context so we see the package being scanned?
  • Can you list the contents of the test/fixtures/testdir5 directory?

sschuberth avatar Aug 14 '24 07:08 sschuberth

I have added parts of the necessary information. Unfortunately the tmp directory is cleared as soon as the job ist processed. So we will retry the run with a sleep.

schvvarzekatze avatar Aug 14 '24 08:08 schvvarzekatze

I've created https://github.com/oss-review-toolkit/ort/pull/8989 to help with debugging this issue. Can you run again with that PR merged?

sschuberth avatar Aug 14 '24 09:08 sschuberth

Thank you then I will rebuild a docker image as soon as that PR is merged and rerun the ort scan.

schvvarzekatze avatar Aug 14 '24 10:08 schvvarzekatze

Hello @sschuberth: I have rerun it with the logging you provided. I hope this helps to clarify.

08:20:14.380 [DefaultDispatcher-worker-16] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/babel/babel.git.
08:20:14.380 [DefaultDispatcher-worker-16] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision 'a7620bd266ae1345975767bbc7abf09034437017' with depth limited to 50.
08:20:16.544 [DefaultDispatcher-worker-16] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout a7620bd266ae1345975767bbc7abf09034437017' in '/tmp/ort-DefaultWorkingTreeCache17766079121285209300'...
08:20:24.021 [DefaultDispatcher-worker-6] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/babel/babel.git.
08:20:24.021 [DefaultDispatcher-worker-6] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision 'e0375121bfea96d277f49543dc543f116523cfd0' with depth limited to 50.
08:20:25.729 [DefaultDispatcher-worker-6] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout e0375121bfea96d277f49543dc543f116523cfd0' in '/tmp/ort-DefaultWorkingTreeCache17766079121285209300'...
08:20:32.877 [DefaultDispatcher-worker-7] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/babel/babel.git.
08:20:32.877 [DefaultDispatcher-worker-7] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision '38c23cded40af3ccc8d2c39dbe165e9b446e55a3' with depth limited to 50.
08:20:34.590 [DefaultDispatcher-worker-7] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout 38c23cded40af3ccc8d2c39dbe165e9b446e55a3' in '/tmp/ort-DefaultWorkingTreeCache17766079121285209300'...
08:25:17.405 [DefaultDispatcher-worker-11] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout f069547c3d25cb9602fdbc1d9eb1f34046dc8353' in '/tmp/ort-DefaultWorkingTreeCache141016[2549](https://git.intra.xyz.com/xyz/project/-/jobs/1486812#L2549)2012631433'...
08:28:44.515 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Created file lists for 1069 provenances in 12m 44.782219038s.
08:28:44.544 [main] WARN  com.zaxxer.hikari.pool.PoolBase - HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@18026052 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
08:28:44.545 [main] WARN  com.zaxxer.hikari.pool.PoolBase - HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@6e068ac9 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
08:28:44.545 [main] WARN  com.zaxxer.hikari.pool.PoolBase - HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@3d0352 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
08:28:44.545 [main] WARN  com.zaxxer.hikari.pool.PoolBase - HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@bb6f3f7 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
08:28:44.545 [main] WARN  com.zaxxer.hikari.pool.PoolBase - HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@388f1258 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
08:28:46.123 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Creating file archives for 2 package(s).
08:28:46.125 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/feross/safe-buffer.git.
08:28:46.125 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision 'e4d9032c56faa8a024dce8b501b4e31d5c2ab300' with depth limited to 50.
08:28:46.614 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout e4d9032c56faa8a024dce8b501b4e31d5c2ab300' in '/tmp/ort-DefaultWorkingTreeCache12522413385701667413'...
08:28:46.856 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader13860422904187057827'...
08:28:46.859 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader13860422904187057827' in 2.297392ms.
08:28:46.887 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader13860422904187057827' to storage in 27.907610ms.
08:28:46.891 [main] INFO  org.eclipse.jgit.lib.Repository - Updating working tree from https://github.com/isaacs/node-tar.git.
08:28:46.891 [main] INFO  org.ossreviewtoolkit.plugins.versioncontrolsystems.git.Git - Trying to fetch only revision 'bef7b1e4ffab822681fea2a9b22187192ed14717' with depth limited to 50.
08:28:48.378 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout bef7b1e4ffab822681fea2a9b22187192ed14717' in '/tmp/ort-DefaultWorkingTreeCache11332046505647301986'...
08:28:48.740 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archiving files matching [license*, patents, readme*] from '/tmp/ort-DefaultProvenanceDownloader2483837061893383424'...
08:28:48.755 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Archived directory '/tmp/ort-DefaultProvenanceDownloader2483837061893383424' in 13.718638ms.
08:28:48.772 [main] INFO  org.ossreviewtoolkit.model.utils.FileArchiver - Wrote archive of directory '/tmp/ort-DefaultProvenanceDownloader2483837061893383424' to storage in 16.965873ms.
08:28:48.795 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Created file archives for 2 package(s) in 2.671656413s.
08:28:50.090 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'scancode --version' in '/home/ort'...
08:28:51.913 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'scancode --version' in '/home/ort'...
Exception in thread "main" java.io.IOException: Could not delete file '/tmp/ort-DefaultWorkingTreeCache593982492075894716/test/fixtures/testdir5'.
	at org.ossreviewtoolkit.utils.common.ExtensionsKt.safeDeleteRecursively(Extensions.kt:151)
	at org.ossreviewtoolkit.utils.common.ExtensionsKt.safeDeleteRecursively$default(Extensions.kt:128)
	at org.ossreviewtoolkit.utils.common.ExtensionsKt.safeDeleteRecursively(Extensions.kt:132)
	at org.ossreviewtoolkit.utils.common.ExtensionsKt.safeDeleteRecursively$default(Extensions.kt:128)
	at org.ossreviewtoolkit.utils.common.ExtensionsKt.safeDeleteRecursively(Extensions.kt:132)
	at org.ossreviewtoolkit.utils.common.ExtensionsKt.safeDeleteRecursively$default(Extensions.kt:128)
	at org.ossreviewtoolkit.utils.common.ExtensionsKt.safeDeleteRecursively(Extensions.kt:132)
	at org.ossreviewtoolkit.utils.common.ExtensionsKt.safeDeleteRecursively$default(Extensions.kt:128)
	at org.ossreviewtoolkit.downloader.DefaultWorkingTreeCache.shutdown(WorkingTreeCache.kt:99)
	at org.ossreviewtoolkit.plugins.commands.scanner.ScannerCommand$runScanners$5.invokeSuspend(ScannerCommand.kt:240)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
	at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:277)
	at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:95)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:69)
	at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:48)
	at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
	at org.ossreviewtoolkit.plugins.commands.scanner.ScannerCommand.runScanners(ScannerCommand.kt:240)
	at org.ossreviewtoolkit.plugins.commands.scanner.ScannerCommand.run(ScannerCommand.kt:142)
	at com.github.ajalt.clikt.parsers.Parser.finalizeAndRun(Parser.kt:348)
	at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:218)
	at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:245)
	at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:42)
	at com.github.ajalt.clikt.core.CliktCommand.parse(CliktCommand.kt:457)
	at com.github.ajalt.clikt.core.CliktCommand.parse$default(CliktCommand.kt:454)
	at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:474)
	at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:481)
	at org.ossreviewtoolkit.cli.OrtMainKt.main(OrtMain.kt:85)
Uploading artifacts for failed job
00:03
Uploading artifacts...
build/oss-review-toolkit/*: found 4 matching artifact files and directories 
Uploading artifacts as "archive" to coordinator... 201 Created  id=1486812 responseStatus=201 Created token=glcbt-64
Cleaning up project directory and file based variables
00:02
ERROR: Job failed: exit code 1

schvvarzekatze avatar Aug 28 '24 09:08 schvvarzekatze

@schvvarzekatze I don't see any of the new debug logging in there. Note that you explicitly have to run ORT with --debug to enable these.

sschuberth avatar Aug 28 '24 09:08 sschuberth

Thank you. I tried it another time with --debug

    run-ort --info scan -i "/xyz-project/${ortDirFromProjectRoot}/analyzer/analyzer-result.yml" -o "/xyz-project/${ortDirFromProjectRoot}/scan/" --skip-excluded || test -f "$scanOutput" --debug

but the logs still look the same.

I do not find

Deleting cached working tree for in the full log history.

schvvarzekatze avatar Aug 29 '24 07:08 schvvarzekatze

The --debug needs to replace the --info you have at the beginning:

run-ort --info scan ...

sschuberth avatar Aug 29 '24 07:08 sschuberth

I just tried it but with a local run (as all logging results could not be collected on gitlab). The error is the same anyway. The logs are in the initial comment.

schvvarzekatze avatar Aug 30 '24 15:08 schvvarzekatze

Note to myself: The new debug logs now indicate that the problem probably is the non-UTF8 filename in https://github.com/fshost/node-dir/tree/master/test/fixtures/testdir5.

sschuberth avatar Sep 02 '24 08:09 sschuberth

Thank you, with the fix the scan runs sucessfully again.

schvvarzekatze avatar Sep 05 '24 15:09 schvvarzekatze