`-Djavax.net.ssl.trustStore` certs not properly passed when downloading deps in BSP

[jiramares]

Versions Scala CLI version: 1.3.2 Scala version (default): 3.4.2

IntelliJ IDEA 2024.1.3 (Ultimate Edition) Build #IU-241.17890.1, built on June 4, 2024

Description We used company artifactory to resolve/cache artefacts. Therefor I have to set up environment varianble COURSIER_REPOSITORIES to handle it right. Because we are using self signed certificate on server there is problem with resolving artefacts directly using coursier (upgrades scala-cli, scala or java).

I realized that there is config repositories.default that seems to me to fulfil the same thing but only for scala-cli (not system wide). So I started using it. It works well but breaks the integration with IDE.

After starting using repositories.default instead of COURSIER_REPOSITORIES I can't build in IDEA via BSP. In BSP window there has been no tree under the root of project. The problem is with our artifactory server with self signed certificate because using maven central causes everything works fine.

To Reproduce

1. scala-cli config repositories.default ivy2local https://our-server/with-self-signed-certificate/repo --power
2. create directory test
3. in it create file test.sc with content

//> using scala 3.3

println(sum(2, 3))

def sum(a: Int, b: Int) = a + b

4. in the directory test run scala-cli setup-ide test.sc
5. open IDEA and File / New / Project from Existing Sources - choose folder test and in following dialog choose model BSP

In BSP window there is no tree and println is marked as uknown symbol.

Expected behaviour There should be dependencies in BSP window and println shoul be known.

[jiramares]

And now it is not working with artifact proxy with self-signed certificate via https at all. Neither COURSIER_REPOSITORIES environment variable nor repositories.default scala-cli config. I don't know what to do :-(

[Gedochao]

I don't know what to do :-(

Hey, I'll try to tackle this soon (likely won't be this week, however). I'd suspect something is wrong with the config sub-command's repositories.* settings, but COURSIER_REPOSITORIES is known and proven to be reliable... 😕
Can't tell you much until I take a closer look.

[jiramares]

Hi @Gedochao, I have been a little more investigating and it looks that problem is with https protocol with artifact proxy (not with its setting via COURSIER_REPOSITORIES or repositories.default). I have to restart computer between tests because I can't reset the environment well (killing all java processes have not been enough). scala-cli works great with http and https too (for https I have to use -Djavax.net.ssl.trustStore). But the IDEA is not able to resolve using https at all (not possible to validate server certificate - I have imported it into IDEA's store - it looks like idea is using coursier under the hood too but I can't specify -Djavax.net.ssl.trustStore). To resolve dependencies is job of IDEA or bloop?

[jiramares]

From my point of view it directs to coursier that is run under the hood to resolve artifacts. But when run via scala-cli I can add -Djavax.net.ssl.trustStore but when run in IDEA it is not possible and therfore not working.

[Gedochao]

To resolve dependencies is job of IDEA or bloop?

Definitely Scala CLI/Bloop, depending on which dependencies. Both use Coursier under the hood. However, I can't be sure what IDEA does in the background.

From my point of view it directs to coursier that is run under the hood to resolve artifacts. But when run via scala-cli I can add -Djavax.net.ssl.trustStore but when run in IDEA it is not possible and therfore not working.

@jiramares Can you raise a corresponding ticket on the JetBrains issue tracker? (https://youtrack.jetbrains.com/issues) So that we can establish this works as expected on the IDEA side of things.

[jiramares]

I have already raised one an the result is - it is not IDEA problem (https://youtrack.jetbrains.com/issue/SCL-22728/scala-cli-project-not-working)](https://youtrack.jetbrains.com/issue/SCL-22728/scala-cli-project-not-working)

[jiramares]

But .. who starts the bloop server, from mine point of view it is started by scala-cli and IDEA in calling it via BSP? Therefore if the bloop is responsible for resolving why it is run okay through scala-cli and not running via IDEA? I am confused....

[Gedochao]

One extra question - have you tried running it in Metals? I just want to confirm this isn't IDEA specific, since from what I understand it works when run outside of IDEA. I'd theorise maybe some coursier instance ran inside of IntelliJ is using a JVM without the cert flag.

I hope we won't end up bouncing this back and forth...

[jiramares]

No, I haven't ... be honest don't know how to make it. Just install Metals into VSC and run scala-cli setup-id . and open it in VSC? Or it is necessary to do something more?

[Gedochao]

No, I haven't ... be honest don't know how to make it. Just install Metals into VSC and run scala-cli setup-id . and open it in VSC? Or it is necessary to do something more?

Exactly this. plus start the Metals server and check if everything works.

[jiramares]

I try to make it but I am sorry not to test it now ... hope to make it this week. j

[jiramares]

I have installed metals into VSC and have same result as in IDEA:

024.06.28 15:33:28 INFO  BSP server: [error]   download error: Caught javax.net.ssl.SSLHandshakeException (PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target) while downloading https://ids/artifactory/repo/org/typelevel/cats-core_3/2.12.0/cats-core_3-2.12.0.pom

Looks like that the -Djavax.net.ssl.trustStore have to be somehow passed to bloop too...

[tgodzik]

@jiramares do you use cats within your project?

[jiramares]

Yes, because of dylemma.xml-spac

[tgodzik]

My guess is that we are not adding the certs when downloading sources somehow, but it seems unexpected :thinking:

[Gedochao]

@jiramares are you still encountering this issue with Scala CLI v1.5.1? I suspect we may have fixed it in #3169.

[jiramares]

It is still not working. If I repeat the steps in issue description I am getting following stacktrace:

java.lang.Exception: Error getting Bloop class path
  bloop.rifle.BloopRifle$.startServer(BloopRifle.scala:51)
  bloop.rifle.BloopServer$.startBloop$1(BloopServer.scala:77)
  bloop.rifle.BloopServer$.ensureBloopRunning(BloopServer.scala:108)
  bloop.rifle.BloopServer$.bsp(BloopServer.scala:156)
  bloop.rifle.BloopServer$.buildServer(BloopServer.scala:186)
  scala.build.bsp.BspImpl.$anonfun$10(BspImpl.scala:424)
  scala.build.compiler.BloopCompiler.<init>(BloopCompiler.scala:15)
  scala.build.bsp.BspImpl.newBloopSession(BspImpl.scala:429)
  scala.build.bsp.BspImpl.run(BspImpl.scala:465)
  scala.cli.commands.bsp.Bsp$.runCommand$$anonfun$1(Bsp.scala:185)
  scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
  scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
  scala.build.bsp.BspThreads$.withThreads(BspThreads.scala:23)
  scala.cli.commands.bsp.Bsp$.runCommand(Bsp.scala:174)
  scala.cli.commands.bsp.Bsp$.runCommand(Bsp.scala:76)
  scala.cli.commands.ScalaCommand.run(ScalaCommand.scala:390)
  scala.cli.commands.ScalaCommand.run(ScalaCommand.scala:371)
  caseapp.core.app.CaseApp.main(CaseApp.scala:166)
  scala.cli.commands.ScalaCommand.main(ScalaCommand.scala:356)
  caseapp.core.app.CommandsEntryPoint.main(CommandsEntryPoint.scala:370)
  scala.cli.ScalaCliCommands.main(ScalaCliCommands.scala:125)
  scala.cli.ScalaCli$.main0(ScalaCli.scala:320)
  scala.cli.ScalaCli$.main(ScalaCli.scala:124)
  scala.cli.ScalaCli.main(ScalaCli.scala)
scala.build.errors.FetchingDependenciesError: Error downloading ch.epfl.scala:bloop-frontend_2.12:2.0.2
  not found: /home/jmares/.ivy2/local/ch.epfl.scala/bloop-frontend_2.12/2.0.2/ivys/ivy.xml
  download error: Caught javax.net.ssl.SSLHandshakeException (PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target) while downloading https://our-server/with-self-signed-certificate/repo/ch/epfl/scala/bloop-frontend_2.12/2.0.2/bloop-frontend_2.12-2.0.2.pom
  scala.build.Artifacts$.toFetchingDependenciesError(Artifacts.scala:723)
  scala.build.Artifacts$.fetchCsDependencies$$anonfun$1$$anonfun$2(Artifacts.scala:697)
  scala.util.Either$LeftProjection.map(Either.scala:622)
  scala.build.Artifacts$.fetchCsDependencies$$anonfun$1(Artifacts.scala:691)
  scala.build.EitherCps$Helper.apply(EitherCps.scala:19)
  scala.build.Artifacts$.fetchCsDependencies(Artifacts.scala:671)
  scala.build.Artifacts$.fetchAnyDependenciesWithResult$$anonfun$1(Artifacts.scala:590)
  scala.build.EitherCps$Helper.apply(EitherCps.scala:19)
  scala.build.Artifacts$.fetchAnyDependenciesWithResult(Artifacts.scala:569)
  scala.build.Artifacts$.fetchAnyDependencies$$anonfun$1(Artifacts.scala:555)
  scala.build.EitherCps$Helper.apply(EitherCps.scala:19)
  scala.build.Artifacts$.fetchAnyDependencies(Artifacts.scala:546)
  scala.build.Artifacts$.artifacts$$anonfun$1(Artifacts.scala:486)
  scala.build.EitherCps$Helper.apply(EitherCps.scala:19)
  scala.build.Artifacts$.artifacts(Artifacts.scala:484)
  scala.build.Bloop$.bloopClassPath$$anonfun$1(Bloop.scala:74)
  scala.build.EitherCps$Helper.apply(EitherCps.scala:19)
  scala.build.Bloop$.bloopClassPath(Bloop.scala:72)
  scala.build.Bloop$.bloopClassPath$$anonfun$2(Bloop.scala:105)
  scala.build.EitherCps$Helper.apply(EitherCps.scala:19)
  scala.build.Bloop$.bloopClassPath(Bloop.scala:95)
  scala.cli.commands.shared.SharedCompilationServerOptions.$anonfun$13(SharedCompilationServerOptions.scala:265)
  bloop.rifle.BloopRifle$.startServer(BloopRifle.scala:50)
  bloop.rifle.BloopServer$.startBloop$1(BloopServer.scala:77)
  bloop.rifle.BloopServer$.ensureBloopRunning(BloopServer.scala:108)
  bloop.rifle.BloopServer$.bsp(BloopServer.scala:156)
  bloop.rifle.BloopServer$.buildServer(BloopServer.scala:186)
  scala.build.bsp.BspImpl.$anonfun$10(BspImpl.scala:424)
  scala.build.compiler.BloopCompiler.<init>(BloopCompiler.scala:15)
  scala.build.bsp.BspImpl.newBloopSession(BspImpl.scala:429)
  scala.build.bsp.BspImpl.run(BspImpl.scala:465)
  scala.cli.commands.bsp.Bsp$.runCommand$$anonfun$1(Bsp.scala:185)
  scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
  scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
  scala.build.bsp.BspThreads$.withThreads(BspThreads.scala:23)
  scala.cli.commands.bsp.Bsp$.runCommand(Bsp.scala:174)
  scala.cli.commands.bsp.Bsp$.runCommand(Bsp.scala:76)
  scala.cli.commands.ScalaCommand.run(ScalaCommand.scala:390)
  scala.cli.commands.ScalaCommand.run(ScalaCommand.scala:371)
  caseapp.core.app.CaseApp.main(CaseApp.scala:166)
  scala.cli.commands.ScalaCommand.main(ScalaCommand.scala:356)
  caseapp.core.app.CommandsEntryPoint.main(CommandsEntryPoint.scala:370)
  scala.cli.ScalaCliCommands.main(ScalaCliCommands.scala:125)
  scala.cli.ScalaCli$.main0(ScalaCli.scala:320)
  scala.cli.ScalaCli$.main(ScalaCli.scala:124)
  scala.cli.ScalaCli.main(ScalaCli.scala)
coursier.error.ResolutionError$CantDownloadModule: Error downloading ch.epfl.scala:bloop-frontend_2.12:2.0.2
  not found: /home/jmares/.ivy2/local/ch.epfl.scala/bloop-frontend_2.12/2.0.2/ivys/ivy.xml
  download error: Caught javax.net.ssl.SSLHandshakeException (PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target) while downloading https://our-server/with-self-signed-certificate/repo/ch/epfl/scala/bloop-frontend_2.12/2.0.2/bloop-frontend_2.12-2.0.2.pom
  coursier.Resolve$.$anonfun$validate$1(Resolve.scala:403)
  scala.collection.immutable.List.map(List.scala:247)
  scala.collection.immutable.List.map(List.scala:79)
  coursier.Resolve$.validate(Resolve.scala:401)
  coursier.Resolve.validate0$1(Resolve.scala:160)
  coursier.Resolve.$anonfun$ioWithConflicts0$4(Resolve.scala:210)
  coursier.util.Task$.$anonfun$flatMap$extension$1(Task.scala:14)
  coursier.util.Task$.$anonfun$flatMap$extension$1$adapted(Task.scala:14)
  coursier.util.Task$.wrap(Task.scala:82)
  coursier.util.Task$.$anonfun$flatMap$2(Task.scala:14)
  scala.concurrent.impl.Promise$Transformation.run(Promise.scala:470)
  [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
  [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
  [email protected]/java.lang.Thread.run(Thread.java:833)
  com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:775)
  com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:203)

it looks like the problem is with coursier that is running its own JDK and we have to somewhere specify -Djavax.net.ssl.trustStore=/opt/jdk/lib/security/cacerts' but I don't know where :-(

PS. The JAVA_HOME jdk and IntelliJ IDEA JDK cacerts contains imported certificate to verify self-signed server certificate.

PS2. I am a little bit confused because the file .scala-build/ide-envs.json contains value: "COURSIER_REPOSITORIES":"ivy2local|http://our-server/with-self-signed-certificate/repo" that is specified using ENV variable COURSIER_REPOSITORIES but coursier loads artefacts from repository specified in scala-cli config.

PS3. If I removed this environment variable the value COURSIER_REPOSITORIES has been missing in .scala-build/ide-envs.json file.

[tgodzik]

Did you do scala-cli setup-ide -Djavax.net.ssl.trustStore=/opt/jdk/lib/security/cacerts . ? It should then use that property while downloading.

[jiramares]

@tgodzik That's it ... it works. Great .... I am going to try to finalize solution based on it.