coursier
coursier copied to clipboard
Published
20 hours ago •
coursier
coursier
Race condition while moving temporary .part files
Parallel builds on our Jenkins CI box frequently fail to resolve dependencies with the following error:
[error] (update) lmcoursier.internal.shaded.coursier.error.FetchError$DownloadingArtifacts: Error fetching artifacts:
[error] https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar: download error: Caught java.nio.file.NoSuchFileException: /Users/bomarchman/Library/Caches/Coursier/v1/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/.sbt-git-javadoc.jar.part -> /Users/bomarchman/Library/Caches/Coursier/v1/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar (/Users/bomarchman/Library/Caches/Coursier/v1/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/.sbt-git-javadoc.jar.part -> /Users/bomarchman/Library/Caches/Coursier/v1/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar) while downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
Coursier is trying to move .sbt-git-javadoc.jar.part to .sbt-git-javadoc.jar, but .sbt-git-javadoc.jar.part no longer exists because another process has already moved it. This only happens when multiple SBT processes are simultaneously trying to resolve the same dependency into a shared cache.
The error seems to originate here. From my reading of the code, it looks like the structure lock is acquired here to download the .part file, then released to download auxiliary files, then acquired again to move the .part file to its final location. Another process could easily grab the lock and move the .part file in between those two locked sections.
Could this be resolved by wrapping all of these lines in the structure lock? Or are there other considerations that require the lock to be released in there? I'm happy to take a stab at this if someone can confirm that this would be a viable solution.
Versions
- Scala: 2.12.11
- SBT: 1.3.1
- Platforms MacOs and Linux
To Reproduce
Create four separate projects named coursier-race-condition-1, coursier-race-condition-2, etc with the following project structure:
build.sbt
scalaVersion := "2.12.11"
project/build.properties
sbt.version = 1.3.13
project/plugins.sbt
addSbtPlugin("com.typesafe.sbt" % "sbt-git" % "1.0.0")
Run the following to start parallel SBT instances in each of these projects
# Clear cached versions of `sbt-git`
rm -rv ~/$CACHE_LOCATION/v1/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/
here=$(pwd)
for i in $(seq 1 4); do
cd "$here/coursier-race-condition-$i" && sbt compile &
done
Full Stacktrace
[error] lmcoursier.internal.shaded.coursier.error.FetchError$DownloadingArtifacts: Error fetching artifacts:
[error] https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar: download error: Caught java.nio.file.NoSuchFileException: /Users/bomarchman/Library/Caches/Coursier/v1/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/.sbt-git-javadoc.jar.part -> /Users/bomarchman/Library/Caches/Coursier/v1/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar (/Users/bomarchman/Library/Caches/Coursier/v1/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/.sbt-git-javadoc.jar.part -> /Users/bomarchman/Library/Caches/Coursier/v1/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar) while downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
[error]
[error] at lmcoursier.internal.shaded.coursier.Artifacts$.$anonfun$fetchArtifacts$14(Artifacts.scala:302)
[error] at lmcoursier.internal.shaded.coursier.util.Task$.$anonfun$flatMap$2(Task.scala:14)
[error] at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:307)
[error] at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:41)
[error] at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] at java.lang.Thread.run(Thread.java:748)
[error] Caused by: lmcoursier.internal.shaded.coursier.cache.ArtifactError$DownloadError: download error: Caught java.nio.file.NoSuchFileException: /Users/bomarchman/Library/Caches/Coursier/v1/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/.sbt-git-javadoc.jar.part -> /Users/bomarchman/Library/Caches/Coursier/v1/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar (/Users/bomarchman/Library/Caches/Coursier/v1/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/.sbt-git-javadoc.jar.part -> /Users/bomarchman/Library/Caches/Coursier/v1/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar) while downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache$.helper$2(FileCache.scala:1018)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache$.coursier$cache$FileCache$$downloading(FileCache.scala:1032)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.doDownload$1(FileCache.scala:320)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.$anonfun$download$54(FileCache.scala:507)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$.loop$1(CacheLocks.scala:59)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$.withLockOr(CacheLocks.scala:84)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.$anonfun$download$32(FileCache.scala:508)
[error] at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
[error] at scala.util.Success.$anonfun$map$1(Try.scala:255)
[error] at scala.util.Success.map(Try.scala:213)
[error] at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
[error] at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
[error] at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
[error] at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] at java.lang.Thread.run(Thread.java:748)
[error] Caused by: java.nio.file.NoSuchFileException: /Users/bomarchman/Library/Caches/Coursier/v1/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/.sbt-git-javadoc.jar.part -> /Users/bomarchman/Library/Caches/Coursier/v1/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
[error] at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
[error] at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
[error] at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:396)
[error] at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262)
[error] at java.nio.file.Files.move(Files.java:1395)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.$anonfun$download$46(FileCache.scala:412)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$$anon$1.call(CacheLocks.scala:22)
[error] at lmcoursier.internal.shaded.coursier.paths.CachePath.withStructureLock(CachePath.java:139)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$.withStructureLock(CacheLocks.scala:22)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.$anonfun$download$33(FileCache.scala:410)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache$.$anonfun$downloading$1(FileCache.scala:998)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$.withUrlLock(CacheLocks.scala:102)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache$.helper$2(FileCache.scala:998)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache$.coursier$cache$FileCache$$downloading(FileCache.scala:1032)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.doDownload$1(FileCache.scala:320)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.$anonfun$download$54(FileCache.scala:507)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$.loop$1(CacheLocks.scala:59)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$.withLockOr(CacheLocks.scala:84)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.$anonfun$download$32(FileCache.scala:508)
[error] at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
[error] at scala.util.Success.$anonfun$map$1(Try.scala:255)
[error] at scala.util.Success.map(Try.scala:213)
[error] at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
[error] at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
[error] at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
[error] at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] at java.lang.Thread.run(Thread.java:748)
Thanks for the report. I was able to reproduce that by doing the following:
$ for i in $(seq 1 4); do
mkdir -p repo-$i
cd repo-$i
mkdir -p project
echo 'sbt.version = 1.3.13' > project/build.properties
echo 'scalaVersion := "2.12.11"' > build.sbt
echo 'addSbtPlugin("com.typesafe.sbt" % "sbt-git" % "1.0.0")' > project/plugins.sbt
cd ..
done
$ export BASE="$(pwd)"
$ export COURSIER_CACHE="$(pwd)/cache"
$ for i in $(seq 1 4); do
cd $BASE/repo-$i && sbt compile && cd -
done
$ rm -rf cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git
$ for i in $(seq 1 4); do
cd $BASE/repo-$i && ( sbt compile </dev/null &>output-$i & ); cd -
done
$ tail -F repo-?/output-?
I've been getting the following output:
Click to expand
==> repo-1/output-1 <==
==> repo-2/output-2 <==
==> repo-3/output-3 <==
==> repo-4/output-4 <==
==> repo-1/output-1 <==
[info] welcome to sbt 1.3.13 (Oracle Corporation Java 1.8.0_121)
==> repo-4/output-4 <==
[info] welcome to sbt 1.3.13 (Oracle Corporation Java 1.8.0_121)
==> repo-2/output-2 <==
[info] welcome to sbt 1.3.13 (Oracle Corporation Java 1.8.0_121)
==> repo-3/output-3 <==
[info] welcome to sbt 1.3.13 (Oracle Corporation Java 1.8.0_121)
==> repo-4/output-4 <==
[info] loading settings for project repo-4-build from plugins.sbt ...
==> repo-1/output-1 <==
[info] loading settings for project repo-1-build from plugins.sbt ...
==> repo-2/output-2 <==
[info] loading settings for project repo-2-build from plugins.sbt ...
==> repo-3/output-3 <==
[info] loading settings for project repo-3-build from plugins.sbt ...
==> repo-4/output-4 <==
[info] loading project definition from /Users/alexandre/projects/test/coursier-issue-1815-2/repo-4/project
==> repo-1/output-1 <==
[info] loading project definition from /Users/alexandre/projects/test/coursier-issue-1815-2/repo-1/project
==> repo-3/output-3 <==
[info] loading project definition from /Users/alexandre/projects/test/coursier-issue-1815-2/repo-3/project
==> repo-2/output-2 <==
[info] loading project definition from /Users/alexandre/projects/test/coursier-issue-1815-2/repo-2/project
==> repo-1/output-1 <==
[warn] sbt-git.jar no longer exists at /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/jars/sbt-git.jar
==> repo-4/output-4 <==
[warn] sbt-git.jar no longer exists at /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/jars/sbt-git.jar
==> repo-2/output-2 <==
[warn] sbt-git.jar no longer exists at /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/jars/sbt-git.jar
==> repo-3/output-3 <==
[warn] sbt-git.jar no longer exists at /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/jars/sbt-git.jar
==> repo-1/output-1 <==
Downloading https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-git_2.12_1.0/1.0.0/sbt-git-1.0.0.pom
==> repo-4/output-4 <==
Downloading https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-git_2.12_1.0/1.0.0/sbt-git-1.0.0.pom
==> repo-2/output-2 <==
Downloading https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-git_2.12_1.0/1.0.0/sbt-git-1.0.0.pom
==> repo-3/output-3 <==
Downloading https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-git_2.12_1.0/1.0.0/sbt-git-1.0.0.pom
==> repo-1/output-1 <==
Downloaded https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-git_2.12_1.0/1.0.0/sbt-git-1.0.0.pom
Downloading https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-git_2.12_1.0/1.0.0/sbt-git-1.0.0.pom.sha1
Downloaded https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-git_2.12_1.0/1.0.0/sbt-git-1.0.0.pom.sha1
Downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/ivys/ivy.xml
==> repo-2/output-2 <==
Downloaded https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-git_2.12_1.0/1.0.0/sbt-git-1.0.0.pom
Downloading https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-git_2.12_1.0/1.0.0/sbt-git-1.0.0.pom.sha1
Downloaded https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-git_2.12_1.0/1.0.0/sbt-git-1.0.0.pom.sha1
Downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/ivys/ivy.xml
==> repo-4/output-4 <==
Downloaded https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-git_2.12_1.0/1.0.0/sbt-git-1.0.0.pom
Downloading https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-git_2.12_1.0/1.0.0/sbt-git-1.0.0.pom.sha1
Downloaded https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-git_2.12_1.0/1.0.0/sbt-git-1.0.0.pom.sha1
Downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/ivys/ivy.xml
==> repo-3/output-3 <==
Downloaded https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-git_2.12_1.0/1.0.0/sbt-git-1.0.0.pom
Downloading https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-git_2.12_1.0/1.0.0/sbt-git-1.0.0.pom.sha1
Downloaded https://repo1.maven.org/maven2/com/typesafe/sbt/sbt-git_2.12_1.0/1.0.0/sbt-git-1.0.0.pom.sha1
Downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/ivys/ivy.xml
==> repo-1/output-1 <==
Downloaded https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/ivys/ivy.xml
==> repo-2/output-2 <==
Downloaded https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/ivys/ivy.xml
Downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/ivys/ivy.xml.sha1
==> repo-4/output-4 <==
Downloaded https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/ivys/ivy.xml
==> repo-2/output-2 <==
Downloaded https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/ivys/ivy.xml.sha1
Downloading https://repo.typesafe.com/typesafe/ivy-releases/com.typesafe.sbt/sbt-git/1.0.0/ivys/ivy.xml
==> repo-3/output-3 <==
Downloaded https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/ivys/ivy.xml
==> repo-2/output-2 <==
Downloaded https://repo.typesafe.com/typesafe/ivy-releases/com.typesafe.sbt/sbt-git/1.0.0/ivys/ivy.xml
Downloading https://repo.typesafe.com/typesafe/ivy-releases/com.typesafe.sbt/sbt-git/1.0.0/ivys/ivy.xml.sha1
Downloaded https://repo.typesafe.com/typesafe/ivy-releases/com.typesafe.sbt/sbt-git/1.0.0/ivys/ivy.xml.sha1
==> repo-1/output-1 <==
Downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/jars/sbt-git.jar
Downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
Downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/srcs/sbt-git-sources.jar
==> repo-4/output-4 <==
Downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/jars/sbt-git.jar
Downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/srcs/sbt-git-sources.jar
Downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
==> repo-1/output-1 <==
Downloaded https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/jars/sbt-git.jar
Downloaded https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/srcs/sbt-git-sources.jar
==> repo-3/output-3 <==
Downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
==> repo-4/output-4 <==
Downloaded https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/jars/sbt-git.jar
Downloaded https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/srcs/sbt-git-sources.jar
==> repo-1/output-1 <==
Downloaded https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
Downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
==> repo-4/output-4 <==
Downloaded https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
Downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar.sha1
==> repo-2/output-2 <==
Downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
==> repo-3/output-3 <==
Downloaded https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
Downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar.sha1
==> repo-4/output-4 <==
Downloaded https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar.sha1
==> repo-1/output-1 <==
Downloaded https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
==> repo-3/output-3 <==
Downloaded https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar.sha1
==> repo-2/output-2 <==
Downloaded https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
==> repo-4/output-4 <==
[error] lmcoursier.internal.shaded.coursier.error.FetchError$DownloadingArtifacts: Error fetching artifacts:
[error] https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar: download error: Caught java.nio.file.NoSuchFileException: /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/.sbt-git-javadoc.jar.part -> /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar (/Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/.sbt-git-javadoc.jar.part -> /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar) while downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
[error]
[error] at lmcoursier.internal.shaded.coursier.Artifacts$.$anonfun$fetchArtifacts$14(Artifacts.scala:302)
[error] at lmcoursier.internal.shaded.coursier.util.Task$.$anonfun$flatMap$2(Task.scala:14)
[error] at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:307)
[error] at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:41)
[error] at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[error] at java.lang.Thread.run(Thread.java:745)
[error] Caused by: lmcoursier.internal.shaded.coursier.cache.ArtifactError$DownloadError: download error: Caught java.nio.file.NoSuchFileException: /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/.sbt-git-javadoc.jar.part -> /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar (/Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/.sbt-git-javadoc.jar.part -> /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar) while downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache$.helper$2(FileCache.scala:1018)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache$.coursier$cache$FileCache$$downloading(FileCache.scala:1032)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.doDownload$1(FileCache.scala:320)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.$anonfun$download$54(FileCache.scala:507)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$.loop$1(CacheLocks.scala:59)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$.withLockOr(CacheLocks.scala:84)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.$anonfun$download$32(FileCache.scala:508)
[error] at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
[error] at scala.util.Success.$anonfun$map$1(Try.scala:255)
[error] at scala.util.Success.map(Try.scala:213)
[error] at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
[error] at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
[error] at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
[error] at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[error] at java.lang.Thread.run(Thread.java:745)
[error] Caused by: java.nio.file.NoSuchFileException: /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/.sbt-git-javadoc.jar.part -> /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
[error] at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
[error] at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
[error] at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:396)
[error] at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262)
[error] at java.nio.file.Files.move(Files.java:1395)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.$anonfun$download$46(FileCache.scala:412)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$$anon$1.call(CacheLocks.scala:22)
[error] at lmcoursier.internal.shaded.coursier.paths.CachePath.withStructureLock(CachePath.java:139)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$.withStructureLock(CacheLocks.scala:22)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.$anonfun$download$33(FileCache.scala:410)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache$.$anonfun$downloading$1(FileCache.scala:998)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$.withUrlLock(CacheLocks.scala:102)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache$.helper$2(FileCache.scala:998)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache$.coursier$cache$FileCache$$downloading(FileCache.scala:1032)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.doDownload$1(FileCache.scala:320)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.$anonfun$download$54(FileCache.scala:507)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$.loop$1(CacheLocks.scala:59)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$.withLockOr(CacheLocks.scala:84)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.$anonfun$download$32(FileCache.scala:508)
[error] at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
[error] at scala.util.Success.$anonfun$map$1(Try.scala:255)
[error] at scala.util.Success.map(Try.scala:213)
[error] at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
[error] at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
[error] at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
[error] at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[error] at java.lang.Thread.run(Thread.java:745)
[error] (update) lmcoursier.internal.shaded.coursier.error.FetchError$DownloadingArtifacts: Error fetching artifacts:
[error] https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar: download error: Caught java.nio.file.NoSuchFileException: /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/.sbt-git-javadoc.jar.part -> /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar (/Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/.sbt-git-javadoc.jar.part -> /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar) while downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore?
==> repo-3/output-3 <==
[error] lmcoursier.internal.shaded.coursier.error.FetchError$DownloadingArtifacts: Error fetching artifacts:
[error] https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar: download error: Caught java.nio.file.NoSuchFileException: /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/.sbt-git-javadoc.jar.part -> /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar (/Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/.sbt-git-javadoc.jar.part -> /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar) while downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
[error]
[error] at lmcoursier.internal.shaded.coursier.Artifacts$.$anonfun$fetchArtifacts$14(Artifacts.scala:302)
[error] at lmcoursier.internal.shaded.coursier.util.Task$.$anonfun$flatMap$2(Task.scala:14)
[error] at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:307)
[error] at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:41)
[error] at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[error] at java.lang.Thread.run(Thread.java:745)
[error] Caused by: lmcoursier.internal.shaded.coursier.cache.ArtifactError$DownloadError: download error: Caught java.nio.file.NoSuchFileException: /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/.sbt-git-javadoc.jar.part -> /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar (/Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/.sbt-git-javadoc.jar.part -> /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar) while downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache$.helper$2(FileCache.scala:1018)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache$.coursier$cache$FileCache$$downloading(FileCache.scala:1032)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.doDownload$1(FileCache.scala:320)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.$anonfun$download$54(FileCache.scala:507)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$.loop$1(CacheLocks.scala:59)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$.withLockOr(CacheLocks.scala:84)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.$anonfun$download$32(FileCache.scala:508)
[error] at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
[error] at scala.util.Success.$anonfun$map$1(Try.scala:255)
[error] at scala.util.Success.map(Try.scala:213)
[error] at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
[error] at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
[error] at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
[error] at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[error] at java.lang.Thread.run(Thread.java:745)
[error] Caused by: java.nio.file.NoSuchFileException: /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/.sbt-git-javadoc.jar.part -> /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
[error] at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
[error] at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
[error] at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:396)
[error] at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262)
[error] at java.nio.file.Files.move(Files.java:1395)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.$anonfun$download$46(FileCache.scala:412)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$$anon$1.call(CacheLocks.scala:22)
[error] at lmcoursier.internal.shaded.coursier.paths.CachePath.withStructureLock(CachePath.java:139)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$.withStructureLock(CacheLocks.scala:22)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.$anonfun$download$33(FileCache.scala:410)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache$.$anonfun$downloading$1(FileCache.scala:998)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$.withUrlLock(CacheLocks.scala:102)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache$.helper$2(FileCache.scala:998)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache$.coursier$cache$FileCache$$downloading(FileCache.scala:1032)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.doDownload$1(FileCache.scala:320)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.$anonfun$download$54(FileCache.scala:507)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$.loop$1(CacheLocks.scala:59)
[error] at lmcoursier.internal.shaded.coursier.cache.CacheLocks$.withLockOr(CacheLocks.scala:84)
[error] at lmcoursier.internal.shaded.coursier.cache.FileCache.$anonfun$download$32(FileCache.scala:508)
[error] at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
[error] at scala.util.Success.$anonfun$map$1(Try.scala:255)
[error] at scala.util.Success.map(Try.scala:213)
[error] at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
[error] at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
[error] at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
[error] at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[error] at java.lang.Thread.run(Thread.java:745)
[error] (update) lmcoursier.internal.shaded.coursier.error.FetchError$DownloadingArtifacts: Error fetching artifacts:
[error] https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar: download error: Caught java.nio.file.NoSuchFileException: /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/.sbt-git-javadoc.jar.part -> /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar (/Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/.sbt-git-javadoc.jar.part -> /Users/alexandre/projects/test/coursier-issue-1815-2/cache/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar) while downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-git/scala_2.12/sbt_1.0/1.0.0/docs/sbt-git-javadoc.jar
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore?
==> repo-1/output-1 <==
[info] loading settings for project repo-1 from build.sbt ...
==> repo-2/output-2 <==
[info] loading settings for project repo-2 from build.sbt ...
==> repo-1/output-1 <==
[info] set current project to repo-1 (in build file:/Users/alexandre/projects/test/coursier-issue-1815-2/repo-1/)
==> repo-2/output-2 <==
[info] set current project to repo-2 (in build file:/Users/alexandre/projects/test/coursier-issue-1815-2/repo-2/)
==> repo-1/output-1 <==
[info] Executing in batch mode. For better performance use sbt's shell
==> repo-2/output-2 <==
[info] Executing in batch mode. For better performance use sbt's shell
==> repo-1/output-1 <==
[success] Total time: 0 s, completed 31 juil. 2020 19:45:49
==> repo-2/output-2 <==
[success] Total time: 0 s, completed 31 juil. 2020 19:45:49
I wasn't able to reproduce that running coursier fetch --sbt-plugin com.typesafe.sbt:sbt-git:1.0.0 -A '*' instead of sbt compile though…
There are already mechanisms to prevent parallel resolutions to conflict (for each file to download, only one of them will do the download, the other just are just watching the download going).
@alexarchambault Yes, I was also unable to cause the error using the coursier CLI directly.
FYI: I am experiencing this error while running Ammonite (which uses Coursier to download artifacts) scripts in parallel on Jenkins.
@scheleaap Do you have some kind of reproduction? Or more details about how it happened? (what kind of script you were running in parallel, which dependencies you were trying to download, etc.) That could be helpful.
@alexarchambault I constructed an example that triggers an error consistently on my laptop. The error is not always exactly the same, as it's caused by a race condition. The actual dependency I'm downloading is not important; I chose Cats because it's a somewhat bigger library that requires a lot of downloads.
Save the following as test.sc:
#!/usr/bin/env amm
import $ivy.`org.typelevel::cats-core:2.3.0`
@main
def main(args: String*) = {
println("Hello")
Thread.sleep(1000) // Not relevant, just to check that the script is executed in parallel.
}
Then run:
chmod +x test.sc
rm -rf ~/.cache/coursier/* && (./resources/scripts/test.sc & ./resources/scripts/test.sc &)
Click to expand an example error
Failed to resolve ivy dependencies:https://repo1.maven.org/maven2/org/typelevel/cats-core_2.13/2.3.0/cats-core_2.13-2.3.0-sources.jar: checksum not found: /home/scheleaap/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-core_2.13/2.3.0/cats-core_2.13-2.3.0-sources.jar
https://repo1.maven.org/maven2/org/typelevel/cats-core_2.13/2.3.0/cats-core_2.13-2.3.0.jar: download error: Caught java.nio.file.NoSuchFileException: /home/scheleaap/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-core_2.13/2.3.0/.cats-core_2.13-2.3.0.jar.part -> /home/scheleaap/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-core_2.13/2.3.0/cats-core_2.13-2.3.0.jar (/home/scheleaap/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-core_2.13/2.3.0/.cats-core_2.13-2.3.0.jar.part -> /home/scheleaap/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-core_2.13/2.3.0/cats-core_2.13-2.3.0.jar) while downloading https://repo1.maven.org/maven2/org/typelevel/cats-core_2.13/2.3.0/cats-core_2.13-2.3.0.jar
https://repo1.maven.org/maven2/org/typelevel/simulacrum-scalafix-annotations_2.13/0.5.1/simulacrum-scalafix-annotations_2.13-0.5.1-sources.jar: download error: Caught java.nio.file.NoSuchFileException: /home/scheleaap/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/simulacrum-scalafix-annotations_2.13/0.5.1/.simulacrum-scalafix-annotations_2.13-0.5.1-sources.jar.part -> /home/scheleaap/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/simulacrum-scalafix-annotations_2.13/0.5.1/simulacrum-scalafix-annotations_2.13-0.5.1-sources.jar (/home/scheleaap/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/simulacrum-scalafix-annotations_2.13/0.5.1/.simulacrum-scalafix-annotations_2.13-0.5.1-sources.jar.part -> /home/scheleaap/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/simulacrum-scalafix-annotations_2.13/0.5.1/simulacrum-scalafix-annotations_2.13-0.5.1-sources.jar) while downloading https://repo1.maven.org/maven2/org/typelevel/simulacrum-scalafix-annotations_2.13/0.5.1/simulacrum-scalafix-annotations_2.13-0.5.1-sources.jar
Failed to resolve ivy dependencies:https://repo1.maven.org/maven2/org/typelevel/cats-kernel_2.13/2.3.0/cats-kernel_2.13-2.3.0.jar: download error: Caught java.nio.file.NoSuchFileException: /home/scheleaap/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-kernel_2.13/2.3.0/.cats-kernel_2.13-2.3.0.jar.part -> /home/scheleaap/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-kernel_2.13/2.3.0/cats-kernel_2.13-2.3.0.jar (/home/scheleaap/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-kernel_2.13/2.3.0/.cats-kernel_2.13-2.3.0.jar.part -> /home/scheleaap/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-kernel_2.13/2.3.0/cats-kernel_2.13-2.3.0.jar) while downloading https://repo1.maven.org/maven2/org/typelevel/cats-kernel_2.13/2.3.0/cats-kernel_2.13-2.3.0.jar
https://repo1.maven.org/maven2/org/typelevel/cats-core_2.13/2.3.0/cats-core_2.13-2.3.0-sources.jar: download error: Caught java.nio.file.NoSuchFileException: /home/scheleaap/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-core_2.13/2.3.0/.cats-core_2.13-2.3.0-sources.jar.part -> /home/scheleaap/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-core_2.13/2.3.0/cats-core_2.13-2.3.0-sources.jar (/home/scheleaap/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-core_2.13/2.3.0/.cats-core_2.13-2.3.0-sources.jar.part -> /home/scheleaap/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-core_2.13/2.3.0/cats-core_2.13-2.3.0-sources.jar) while downloading https://repo1.maven.org/maven2/org/typelevel/cats-core_2.13/2.3.0/cats-core_2.13-2.3.0-sources.jar
We're experiecing this very frequently (roughly 70% of our builds) when running inside of AWS CodeBuild, but we have never had this happen when running locally or in our Jenkins setup.
Are there any known workarounds at this point?
I saw this error a few times building my sbt plugin in https://github.com/dwickern/sbt-swagger-play/pull/8.
I'm using sbt-projectmatrix to run sbt scripted tests against several versions of different libraries. My current configuration has 24 sbt instances running within a short period of time, with some number >1 running in parallel.
Here's a link to one of the failing builds: https://github.com/dwickern/sbt-swagger-play/pull/8/checks?check_run_id=2375971235
[info] [info] loading settings for project sbt_830ce090-build from plugins.sbt ...
[info] [info] loading project definition from /tmp/sbt_830ce090/project
[info] [info] loading settings for project root from build.sbt ...
[info] [info] set current project to root (in build file:/tmp/sbt_830ce090/)
[info] [error] java.nio.file.NoSuchFileException: /home/runner/.cache/coursier/v1/https/repo1.maven.org/maven2/org/apache/httpcomponents/httpcomponents-parent/10/.httpcomponents-parent-10.pom__sha1.computed1437468769051299379.tmp -> /home/runner/.cache/coursier/v1/https/repo1.maven.org/maven2/org/apache/httpcomponents/httpcomponents-parent/10/.httpcomponents-parent-10.pom__sha1.computed
[info] [error] at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
[info] [error] at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
[info] [error] at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:396)
[info] [error] at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262)
[info] [error] at java.nio.file.Files.move(Files.java:1395)
[info] [error] at lmcoursier.internal.shaded.coursier.cache.FileCache$.coursier$cache$FileCache$$persistedDigest(FileCache.scala:1109)
[info] [error] at lmcoursier.internal.shaded.coursier.cache.FileCache.$anonfun$validateChecksum$4(FileCache.scala:684)
[info] [error] at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
[info] [error] at scala.util.Success.$anonfun$map$1(Try.scala:255)
[info] [error] at scala.util.Success.map(Try.scala:213)
[info] [error] at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
[info] [error] at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
[info] [error] at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
[info] [error] at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
[info] [error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[info] [error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[info] [error] at java.lang.Thread.run(Thread.java:748)
[info] [error] (update) java.nio.file.NoSuchFileException: /home/runner/.cache/coursier/v1/https/repo1.maven.org/maven2/org/apache/httpcomponents/httpcomponents-parent/10/.httpcomponents-parent-10.pom__sha1.computed1437468769051299379.tmp -> /home/runner/.cache/coursier/v1/https/repo1.maven.org/maven2/org/apache/httpcomponents/httpcomponents-parent/10/.httpcomponents-parent-10.pom__sha1.computed
[info] [error] Total time: 4 s, completed Apr 18, 2021 9:10:26 PM
The Pants project is seeing a similar race condition consistently in CI with JDK downloads: https://github.com/pantsbuild/pants/issues/12293 (example CI failure: https://github.com/pantsbuild/pants/runs/3010851715?check_suite_focus=true#step:11:919)
Any news on this? It looks like there are lots of different toolchains which run into this issue: mill, sbt, pants, ... This is essentially THE blocker to make parallel processing in mill the default.
I tried to look into coursier source code, but honestly my estimated/reserved time to understand and fiddle with the code was too short. It's probably the same for others. Maybe we can better help @alexarchambault by providing an easy to execute reproducer, which can be integrated in the test suite of coursier.
Issue https://github.com/coursier/coursier/issues/2022 is probably related, and provides some details and some workaround.
Hi,
I'm experiencing this issue. For more details, see: https://github.com/scalameta/scalafmt/issues/2793
"Ugly fixed" here: https://github.com/ghostdogpr/caliban/pull/1072
Hi! Any updates on this issue? Facing the same errors in our parallel CI builds that use "shared" coursier cache.
This is still an issue, so I did a little blackbox investigation. I ran the ammonite script above and traced system calls to watch the actual actions on the file system.
Here is an example output. The file simulacrum-scalafix-annotations_2.13-0.5.1.jar.part is the file causing the error. My interpretation is that the two processes (the PID is the first column) both lock the structure lock file, then open the file simulacrum-scalafix-annotations_2.13-0.5.1.jar.part file, then BOTH write to it, then both take the structure lock and rename it, then release the lock. Perhaps the check to see if the file exists should be done should be done within the structure lock?
785921 1656536140.275762 close(31) = 0
785921 1656536140.276376 write(35, "PK\3\4\24\0\10\10\10\0\0\0!\0\0\0\0\0\0\0\0\0\0\0\0\0\24\0\r\0ME"..., 1371) = 1371
785911 1656536140.292578 openat(AT_FDCWD, "/home/oliver/.cache/coursier/v1/.structure.lock", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 31
785911 1656536140.292737 fcntl(31, F_SETLKW, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0
785911 1656536140.292943 openat(AT_FDCWD, "/home/oliver/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/simulacrum-scalafix-annotations_2.13/0.5.1/.simulacrum-scalafix-annotations_2.13-0.5.1.jar.part", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 36
785911 1656536140.293055 fcntl(31, F_SETLK, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0
785911 1656536140.293114 close(31) = 0
785911 1656536140.293681 write(36, "PK\3\4\24\0\10\10\10\0\0\0!\0\0\0\0\0\0\0\0\0\0\0\0\0\24\0\r\0ME"..., 1371) = 1371
785916 1656536140.312271 write(32, "y\373\272\37\7\373\310\30g\24\333\240\302\34\262m)\275[0\263q\263\222\34\344\fn\241\35H\317"..., 1371) = 1371
--
785908 1656536140.484109 write(34, "m\341\35\273\365C\336\325\3667\354q\n\265{\365\274\237\352\311y\334~?\262\323b\0yi\362\367"..., 1371) = 1371
785908 1656536140.484804 write(34, "=\260\312(\314\226\20\312\334\210\304\5\235\16Z\0106$&\22\4c,,\206\276\27#\240p\371\321"..., 1371) = 1371
785909 1656536140.486278 openat(AT_FDCWD, "/home/oliver/.cache/coursier/v1/.structure.lock", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 31
785909 1656536140.486443 fcntl(31, F_SETLKW, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0
785909 1656536140.486652 openat(AT_FDCWD, "/home/oliver/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/simulacrum-scalafix-annotations_2.13/0.5.1/.simulacrum-scalafix-annotations_2.13-0.5.1.jar.part", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 35
785909 1656536140.486785 fcntl(31, F_SETLK, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0
785909 1656536140.486858 close(31) = 0
785909 1656536140.487500 write(35, "PK\3\4\24\0\10\10\10\0\0\0!\0\0\0\0\0\0\0\0\0\0\0\0\0\24\0\r\0ME"..., 1371) = 1371
785909 1656536140.487812 write(35, "\376\276e\377\251\240\242\214s\305@f\336\376\344\370\233\273\372\21\5(\203\317\340\323xF8\333\360\337"..., 1371 <unfinished ...>
--
785911 1656536140.511067 close(30) = 0
785911 1656536140.511334 rename("/home/oliver/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/simulacrum-scalafix-annotations_2.13/0.5.1/..simulacrum-scalafix-annotations_2.13-0.5.1.jar__sha1.part", "/home/oliver/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/simulacrum-scalafix-annotations_2.13/0.5.1/.simulacrum-scalafix-annotations_2.13-0.5.1.jar__sha1") = 0
785911 1656536140.511550 openat(AT_FDCWD, "/home/oliver/.cache/coursier/v1/.structure.lock", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 30
785911 1656536140.511701 fcntl(30, F_SETLKW, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0
785911 1656536140.511879 rename("/home/oliver/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/simulacrum-scalafix-annotations_2.13/0.5.1/.simulacrum-scalafix-annotations_2.13-0.5.1.jar.part", "/home/oliver/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/simulacrum-scalafix-annotations_2.13/0.5.1/simulacrum-scalafix-annotations_2.13-0.5.1.jar") = 0
785911 1656536140.511951 fcntl(30, F_SETLK, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0
785911 1656536140.512017 close(30) = 0
785911 1656536140.512230 openat(AT_FDCWD, "/home/oliver/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/simulacrum-scalafix-annotations_2.13/0.5.1/.simulacrum-scalafix-annotations_2.13-0.5.1.jar.checked", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 30
785911 1656536140.512324 close(30) = 0
--
785909 1656536140.695310 close(29) = 0
785909 1656536140.695555 rename("/home/oliver/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/simulacrum-scalafix-annotations_2.13/0.5.1/..simulacrum-scalafix-annotations_2.13-0.5.1.jar__sha1.part", "/home/oliver/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/simulacrum-scalafix-annotations_2.13/0.5.1/.simulacrum-scalafix-annotations_2.13-0.5.1.jar__sha1") = 0
785909 1656536140.695764 openat(AT_FDCWD, "/home/oliver/.cache/coursier/v1/.structure.lock", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 29
785909 1656536140.695901 fcntl(29, F_SETLKW, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0
785909 1656536140.696139 rename("/home/oliver/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/simulacrum-scalafix-annotations_2.13/0.5.1/.simulacrum-scalafix-annotations_2.13-0.5.1.jar.part", "/home/oliver/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/simulacrum-scalafix-annotations_2.13/0.5.1/simulacrum-scalafix-annotations_2.13-0.5.1.jar") = -1 ENOENT (No such file or directory)
785909 1656536140.696236 fcntl(29, F_SETLK, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0
785909 1656536140.696298 close(29) = 0
785909 1656536140.696455 fcntl(28, F_SETLK, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0
785909 1656536140.696513 close(28) = 0
In order to recreate this, you can use the following scripts on linux: create run.sh:
rm -rf ~/.cache/coursier/ && (amm test.sc & amm test.sc &)
then test.sc as described by scheleaap
import $ivy.`org.typelevel::cats-core:2.3.0`
@main
def main(args: String*) = {
println("Hello")
Thread.sleep(1000) // Not relevant, just to check that the script is executed in parallel.
}
then run using strace:
strace -f -ttt -e trace=openat,close,write,fsync,rename,renameat,renameat2,flock,fcntl -o strace.out bash ./run.sh
Then to get just the bits related to the file in question:
grep -C 4 "<name of file that is missing>" strace.out
If there is any other investigation I can do to help fix this, please do ask, as this bug is slowing our CI pipelines down considerably because the shared caching in jenkins is effectively broken if using parallelism within mill
If there is any other investigation I can do to help fix this, please do ask, as this bug is slowing our CI pipelines down considerably because the shared caching in jenkins is effectively broken if using parallelism within mill
@ollyw I'm curious which Mill version you are using. I thought, we applied an effective workaround in Mill 0.10.2. (https://github.com/com-lihaoyi/mill/pull/1772) It's not a fix, though, but it should greatly improve the situation when using Mill with parallelism.
@lefou, thanks for responding. We are using mill 0.10.4, although the workarounds for the problem in the CI pipeline have been around since 0.10.1. was used. The person who put them in place has left the project and so I don't know if 0.10.2 was tested. I'll remove the workarounds and report back as to whether 0.10.4 is having this issue.
It seems like a lot of downstream libraries and projects are working around this bug.
It seems like a lot of downstream libraries and projects are working around this bug.
Yeah, that is very unfortunate. But I tried to fix it in coursier myself but didn't get it right in time and no response to my questions. Also dev response to this issue was way to rare, so I perfectly understand why downstream projects chose to work around. Same decision I made in Mill.
@lefou when testing with Mill 0.10.4 without the workarounds some errors reappear:
09:39:46 <mill-module-one>.macro.scalaCompilerClasspath Failed to load dependencies
09:39:46 checksum not found: /root/.cache/coursier/v1/https/<internal-host>/oss-maven/org/scala-lang/scala-library/2.13.8/scala-library-2.13.8.jar
09:39:46 <mill-module-two>.scalaCompilerClasspath Failed to load dependencies
09:39:46 checksum not found: /root/.cache/coursier/v1/https/<internal-host>/oss-maven/org/scala-lang/scala-library/2.13.8/scala-library-2.13.8.jar
Is it possible that the workaround doesn't fix the bootstrapping of the scala compiler, and only works on module dependencies?
@lefou when testing with Mill 0.10.4 without the workarounds some errors reappear:
09:39:46 <mill-module-one>.macro.scalaCompilerClasspath Failed to load dependencies 09:39:46 checksum not found: /root/.cache/coursier/v1/https/<internal-host>/oss-maven/org/scala-lang/scala-library/2.13.8/scala-library-2.13.8.jar 09:39:46 <mill-module-two>.scalaCompilerClasspath Failed to load dependencies 09:39:46 checksum not found: /root/.cache/coursier/v1/https/<internal-host>/oss-maven/org/scala-lang/scala-library/2.13.8/scala-library-2.13.8.jarIs it possible that the workaround doesn't fix the bootstrapping of the scala compiler, and only works on module dependencies?
What we currently don't control is the bootstrapping of ammonite scripts, which means all special $ivy imports in the build.sc. But the scalaCompilerClasspath is actually using CoursierModule.resolveDeps, which is using our workaround. It's worth mentioning that we really only work around those concurrency issues (reported here). Especially, we don't repeat each coursier resolution which failed based on some other issues, but not for concurrency reasons. This still happens quite often especially when on GitHub Actions CI, but for some checksum issues.
It is very uncommon to have checksum issues almost every time with some kind of network corruption, given that most traffic is TLS which guarantees data integrity. My gut feeling is that it is more likely that two coursier writers are writing to the same file. I see nothing in the previous strace that excludes that possibility. I can't see use of O_EXCL, and the structure lock is released before the file being written to is closed.
In fact, the above strace shows two different processes writing to simulacrum-scalafix-annotations_2.13-0.5.1.jar.part. I wonder if there are some bandwidth efficiency saving to be had here if multiple processes are attempting to download the same file in parallel
I think this is an unrelated issue, as I often see these checksum failures in Mill CI without parallel builds. I just wanted to point out, that we don't retry coursier resolution in general, but only retry when we specifically detect a concurrent download event which would fail otherwise with the above listed error messages.
If I get some time I will try and recreate the mill issues on my local machine and trace the output to see what is happening. Is it worth creating a ticket in the mill github repo for this checksum issue? Perhaps you would be more suited to do so, as you have more context of the issue then I do.
I create an issue in Mill to track the sha1 signature issue somewhere.
- https://github.com/com-lihaoyi/mill/issues/1910
Yeah, that is very unfortunate. But I tried to fix it in coursier myself but didn't get it right in time and no response to my questions. Also dev response to this issue was way to rare, so I perfectly understand why downstream projects chose to work around. Same decision I made in Mill.
It's perfectly fine to use workarounds until upstream issues are fixed!
Not sure when I'll have time for it, but @ollyw's comment https://github.com/coursier/coursier/issues/1815#issuecomment-1170502853 should be very helpful to help future investigations.
FWIW, I can't reproduce the error seen in https://github.com/coursier/coursier/issues/1815#issuecomment-736738585 with Ammonite 2.5.5-15-277624cf, while I could on almost all attempts with 2.3.8 (Ammonite version at the time) or 2.5.5. 2.5.5-15-277624cf depends on a newer version of coursier, while 2.5.5 and before were on coursier pre-2.0.0…
$ AMM_VERSION=2.5.5-15-277624cf
$ cs bootstrap ammonite:$AMM_VERSION --ttl 0 -o amm-$AMM_VERSION
$ (rm -rf tmp-cache; export COURSIER_CACHE="$(pwd)/tmp-cache"; ./amm-2.5.5-15-277624cf --help >/dev/null; ./amm-$$AMM_VERSION amm.sc &; ./amm-$AMM_VERSION amm.sc)
(Re-run the last command several times in a row just in case, but I didn't see it fail with 2.5.5-15-277624cf.)
@rlmv Do you still run into that kind of thing using recent versions of sbt? You mention sbt 1.3.1 (which depends on coursier 2.0.0-RC3-4), many things changed since…
@alexarchambault I stopped working on the project that was causing issues a year and half ago so unfortunately I don't have any updates. Great to see the PR with the fix, hopefully other people can confirm that it solves their issues.