mill Use custom JVM versions during Scala compilation, avoid forking for Java compilation

Follow up from https://github.com/com-lihaoyi/mill/issues/3480

Currently JavaModule#compile with a custom JVM relies on Zinc forking a JVM for each Java compilation, which is slow, and Zinc does not support forking JVMs for ScalaModule#compile and it would anyway be prohibitively slow (upstream discussion https://github.com/sbt/zinc/discussions/1498).

The correct solution for this is to move ZincWorkerImpl into a long-lived worker subprocess when def javaHome is overridden. This will allow us to keep the Java compilation process warm so it can benefit from the fast in-memory compiles, while also supporting Scala compilation on custom JVM versions

This will likely require an overhaul of the ZincWorkerApi.scala to dis-entangle things sufficiently that we can impose a subprocess boundary between the caller and callee. This would break binary compatibility and would need to wait for 0.13.0

Nov 18 '24 01:11 lihaoyi

The mentioned relevant Zinc issue: https://github.com/sbt/zinc/discussions/1498

Nov 19 '24 17:11 roman-mibex-2

Taking this off 0.13.0 for now. Would be nice to fix it before then, but can be done in a binary compatible manner if we're careful

Apr 05 '25 19:04 lihaoyi

Unassigning Jakob since he won't have time to get to this

Apr 07 '25 15:04 lihaoyi

Now that https://github.com/com-lihaoyi/mill/pull/5327 has landed, this should be doable purely in the implementation of JvmWorkerImpl without changing the public API

Jun 15 '25 16:06 lihaoyi

I'm taking this on.

Jul 14 '25 11:07 arturaz