Scala3 compiler crashes when cats-effect library is added as a dependency

[meisam]

Steps to reproduce the crash:

1. Add dependencies to cats-effect:

maven_install(
    artifacts = [
        "org.typelevel:cats-free_3:2.8.0",
        "org.typelevel:cats-core_3:2.8.0",
    ],
    repositories = [
        "https://repo1.maven.org/maven2",
    ],
)

2. import the cats library in a scala file.
3. Set the SCALA_VERIOS to 3 in scala_rules (eg. 3.1.0, or 3.1.1, or 3.1.2).
4. Build the project with bazel's scala_rules.

The Scala3 compiler, dotty, crashes with:

exception occurred while compiling path/to/ReproDotytCrash.scala
scala.MatchError: val <none> (of class dotty.tools.dotc.core.Symbols$NoSymbol$) while compiling 
path/to/ReproDotytCrash.scala
val <none> (of class dotty.tools.dotc.core.Symbols$NoSymbol$)
scala.MatchError: val <none> (of class dotty.tools.dotc.core.Symbols$NoSymbol$)
        at dotty.tools.dotc.core.Types$ThisType.cls(Types.scala:2770)
        at dotty.tools.dotc.core.Types$ThisType.underlying(Types.scala:2775)
        at dotty.tools.dotc.core.Types$Type.goThis$1(Types.scala:806)
        at dotty.tools.dotc.core.Types$Type.go$1(Types.scala:700)
        at dotty.tools.dotc.core.Types$Type.findMember(Types.scala:870)
        at dotty.tools.dotc.core.Types$Type.memberBasedOnFlags(Types.scala:666)
        at dotty.tools.dotc.core.Types$Type.member(Types.scala:650)
        at dotty.tools.dotc.core.Types$NamedType.memberDenot(Types.scala:2269)
        at dotty.tools.dotc.core.Types$NamedType.memberDenot(Types.scala:2256)
        at dotty.tools.dotc.core.Types$NamedType.fromDesignator$1(Types.scala:2212)
        at dotty.tools.dotc.core.Types$NamedType.computeDenot(Types.scala:2236)
        at dotty.tools.dotc.core.Types$NamedType.denot(Types.scala:2191)
        at dotty.tools.dotc.core.Types$NamedType.computeSymbol(Types.scala:2148)
        at dotty.tools.dotc.core.Types$NamedType.symbol(Types.scala:2141)
        at dotty.tools.dotc.core.Types$Type.typeSymbol(Types.scala:484)
        at dotty.tools.dotc.core.tasty.TreeUnpickler.readAnnot$$anonfun$2$$anonfun$1(TreeUnpickler.scala:701)
        at dotty.tools.dotc.core.Annotations$.dotty$tools$dotc$core$Annotations$$anon$3$$_$$lessinit$greater$$anonfun$3(Annotations.scala:165)
        at dotty.tools.dotc.core.Annotations$LazyAnnotation.symbol(Annotations.scala:64)
        at dotty.tools.dotc.core.Annotations$Annotation.matches(Annotations.scala:22)
        at dotty.tools.dotc.core.SymDenotations$SymDenotation.dropOtherAnnotations(SymDenotations.scala:283)
        at dotty.tools.dotc.core.SymDenotations$SymDenotation.getAnnotation(SymDenotations.scala:245)
        at dotty.tools.dotc.core.SymDenotations$SymDenotation.targetName(SymDenotations.scala:526)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleExternalRef$1(TreePickler.scala:199)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleNewType(TreePickler.scala:220)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleType(TreePickler.scala:160)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleExternalRef$1(TreePickler.scala:200)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleNewType(TreePickler.scala:220)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleType(TreePickler.scala:160)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleTree(TreePickler.scala:375)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleTree$$anonfun$38$$anonfun$1(TreePickler.scala:428)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
        at scala.collection.immutable.List.foreach(List.scala:333)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleTree$$anonfun$3(TreePickler.scala:428)
        at dotty.tools.dotc.core.tasty.TreePickler.withLength(TreePickler.scala:58)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleTree(TreePickler.scala:429)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleTreeUnlessEmpty(TreePickler.scala:319)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleDef$$anonfun$1(TreePickler.scala:335)
        at dotty.tools.dotc.core.tasty.TreePickler.withLength(TreePickler.scala:58)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleDef(TreePickler.scala:337)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleTree(TreePickler.scala:545)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleTree$$anonfun$40$$anonfun$1(TreePickler.scala:469)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
        at scala.collection.immutable.List.foreach(List.scala:333)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleTree$$anonfun$9(TreePickler.scala:469)
        at dotty.tools.dotc.core.tasty.TreePickler.withLength(TreePickler.scala:58)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleTree(TreePickler.scala:469)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleTreeUnlessEmpty(TreePickler.scala:319)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleDef$$anonfun$1(TreePickler.scala:335)
        at dotty.tools.dotc.core.tasty.TreePickler.withLength(TreePickler.scala:58)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleDef(TreePickler.scala:337)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleTree(TreePickler.scala:560)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleStats$$anonfun$2(TreePickler.scala:360)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
        at scala.collection.immutable.List.foreach(List.scala:333)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleStats(TreePickler.scala:360)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleTree$$anonfun$26(TreePickler.scala:586)
        at dotty.tools.dotc.core.tasty.TreePickler.withLength(TreePickler.scala:58)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleTree(TreePickler.scala:587)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleDef$$anonfun$1(TreePickler.scala:332)
        at dotty.tools.dotc.core.tasty.TreePickler.withLength(TreePickler.scala:58)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleDef(TreePickler.scala:337)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleTree(TreePickler.scala:562)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleStats$$anonfun$2(TreePickler.scala:360)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
        at scala.collection.immutable.List.foreach(List.scala:333)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleStats(TreePickler.scala:360)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleTree$$anonfun$29(TreePickler.scala:602)
        at dotty.tools.dotc.core.tasty.TreePickler.withLength(TreePickler.scala:58)
        at dotty.tools.dotc.core.tasty.TreePickler.pickleTree(TreePickler.scala:602)
        at dotty.tools.dotc.core.tasty.TreePickler.pickle$$anonfun$1(TreePickler.scala:773)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
        at scala.collection.immutable.List.foreach(List.scala:333)
        at dotty.tools.dotc.core.tasty.TreePickler.pickle(TreePickler.scala:773)
        at dotty.tools.dotc.transform.Pickler.run$$anonfun$3$$anonfun$2(Pickler.scala:69)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
        at scala.collection.immutable.List.foreach(List.scala:333)
        at dotty.tools.dotc.transform.Pickler.run$$anonfun$1(Pickler.scala:106)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
        at scala.collection.immutable.List.foreach(List.scala:333)
        at dotty.tools.dotc.transform.Pickler.run(Pickler.scala:106)
        at dotty.tools.dotc.core.Phases$Phase.runOn$$anonfun$1(Phases.scala:308)
        at scala.collection.immutable.List.map(List.scala:246)
        at dotty.tools.dotc.core.Phases$Phase.runOn(Phases.scala:309)
        at dotty.tools.dotc.transform.Pickler.runOn(Pickler.scala:111)
        at dotty.tools.dotc.Run.runPhases$4$$anonfun$4(Run.scala:261)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
        at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1323)
        at dotty.tools.dotc.Run.runPhases$5(Run.scala:272)
        at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:280)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
        at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:68)
        at dotty.tools.dotc.Run.compileUnits(Run.scala:289)
        at dotty.tools.dotc.Run.compileSources(Run.scala:222)
        at dotty.tools.dotc.Run.compile(Run.scala:206)
        at dotty.tools.dotc.Driver.doCompile(Driver.scala:39)
        at io.bazel.rulesscala.scalac.ScalacWorker3.compileScalaSources(ScalacWorker3.java:276)
        at io.bazel.rulesscala.scalac.ScalacWorker3.work(ScalacWorker3.java:83)
        at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:86)
        at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:39)
        at io.bazel.rulesscala.scalac.ScalacWorker3.main(ScalacWorker3.java:40)
Target //:repro-dotty-crash failed to build
INFO: Elapsed time: 0.578s, Critical Path: 0.43s
INFO: 2 processes: 2 internal.
FAILED: Build did NOT complete successfully

[meisam]

I investigated this issue a bit and I have a hunch why it might be happening. It looks like ScalaWorker3.java, which is the class that packs the arguments for the Scala3 compiler and calls the compiler, is called with Scala2 libraries. To be more specific, if I patch ScalaWorker3.java and print the ScalaVersion during the build, i get scala version: 2.13.5.

diff --git a/src/java/io/bazel/rulesscala/scalac/ScalacWorker3.java b/src/java/io/bazel/rulesscala/scalac/ScalacWorker3.java
index 370be29..ef136ee 100644
--- a/src/java/io/bazel/rulesscala/scalac/ScalacWorker3.java
+++ b/src/java/io/bazel/rulesscala/scalac/ScalacWorker3.java
@@ -4,0 +5 @@ import scala.Tuple2;
+import scala.util.Properties;
@@ -75,0 +79 @@ class ScalacWorker3 implements Worker.Interface {
+        System.out.println("scala version: " + Properties.versionNumberString());

The compiler runs in the same process as ScalaWorker3.java, it is just a call to the doCompile method:

  Compiler compiler = driver.newCompiler(r._2);
  
  Reporter reporter = driver.doCompile(compiler, r._1, r._2);

It makes sense that a new process is not spawn for each new compile task, so pay the startup and warmup cost of the scala compiler once, but it sounds like the reported scala version at runtime should match the expected scala3 version.

Cats-effect is an involved library and exercises advanced features in the scala compiler. I thinks that's why this crash does not happen with all the libraries.

I'll do more tests and report the findings.

[fabianhjr]

Got a reproducible nix + bazel build

https://github.com/LibreCybernetics/Monorepo/tree/ref-bazelbuild-scala-3-typelevel-issue

0. Checkout the above branch ( ref-bazelbuild-scala-3-typelevel-issue )
1. nix develop --ignore-environment
2. bazel build //code/lib/toml/core:toml-core

vs sbt

1. cd code/
2. sbt "toml-coreJVM/compile"

Relating to the comment on https://github.com/bazelbuild/rules_scala/pull/1465#issuecomment-1406636236