Make `CompilationResult` more generic

[lefou]

Curent Limitations

Current CompilationResult is very specific for the zinc compiler.

case class CompilationResult(analysisFile: os.Path, classes: PathRef)

It's hard to integrate other compilers easily into Mill:

• Ajc, the AspectJ compiler does not have a analysis file, we need to provide a dummy file

• Kontlinc, the Kotlin compiler also has incremental compilation support, although it's currently not clear how that can be enabled in Mill, but I asssume it's via some analysis file, but with a different format that zinc

There are also other shortcomings:

• When remote caching is used, compiler warnings aren't preserved. It would be nice to include them in the compile result

Solution

Mae ComilationResult extensible, so that additional compiler specific details can be attached.

[lihaoyi]

Do they even need to be the same type? It feels to me like those can be completely different case classes unless there's some reason we want to abstract over them, and it's not clear to me what that reason is

[lefou]

The reason is the shared parent module JavaModule, which allows a KotlinModule to integrate in BSP, IDEA and all other workflows seamlessly. After all, it's just another compiler but everything else is the same.

[lefou]

It is completely transparent for a ScalaModule, whether it's moduleDeps is a Java, Scala, Kotlin, or Something module. But we need to trigger compilation and we need to be able to consume the compilation output (compile.dest/classes). If we now also can extract the incremental info, we can also optimize compilation speed, which is the case when using zinc, but currently not when using other compilers.

[lihaoyi]

got it, that makes sense

[lefou]

We have various ways to implement this:

1. Inheritance
2. Composition

1. Inheritance

trait CompileResult {
  classes: PathRef
}

case class CompilationResult(analysisFile: os.Path, classes: PathRef) extends CompileResult

// ScalaModule
def compile: T[CompiationResult]

trait KotlinIncrDetails {
  // ...
}

class KotlinCompileResult extends CompileResult with KotlinIncrDetails

// KotlinModule
def compile: T[CompileResult with KotlinIncrDetails]
// or
def compile: T[KotlinCompileResult]

2. Composition

case class CompilationResult(
  analysisFile: os.Path, // for compatibility
  classes: PathRef,
  extras: Map[String, AnyRef] = Map()
)

// KotlinModule
def compile: T[CompilationResult] = {
  // ...
  CompilationResult(
    dummyFile, // what we do this currently
    T.dest / "classes",
    extras = Map("kotlinIncrDetails", details)
  )
}

I think I like 1. Inheritance more, as it might better play with upickle. But it requires a bit of discipline to use not the same def names for different Compilers, so we can better encode multiple compilers into one result (for what ever reason, e.g. an aspect compiler still able to provide incremental details).

[lihaoyi]

Inheritance with uPickle will only work if the trait is sealed. If you want jt to be extensible from downstream third party modules, you'll have to use some kind of composition

[lefou]

As part of the discussion of a PR experimenting with the new scalac/zinc pipelining feature (https://github.com/com-lihaoyi/mill/pull/3202), we had the idea of realizing compilation with a set of (co-working) targets and a worker, encapsulating some compiler internals. See comment https://github.com/com-lihaoyi/mill/pull/3202#issuecomment-2165293750 and following.