Occasional `Error loading tasty` and other incremental compilation failures since Scala 3 upgrade

[lihaoyi]

For example

[09488] [info] compiling 1 Scala source to /Users/lihaoyi/Github/mill/out/contrib/gitlab/test/compile.dest/classes ...
[09484] [error] -- Error: /Users/lihaoyi/Github/mill/contrib/scalapblib/test/src/mill/contrib/scalapblib/TutorialTests.scala:29:36 
[09484] [error] 29 |    lazy val millDiscover = Discover[this.type]
[09484] [error]    |                            ^^^^^^^^^^^^^^^^^^^
[09484] [error]    |error while loading Worker,
[09484] [error]    |/Users/lihaoyi/Github/mill/out/core/define/compile.dest/classes/mill/define/Worker.tasty
[09484] [error] one error found
[09486] [error] error while loading Worker,
[09486] [error] /Users/lihaoyi/Github/mill/out/core/define/compile.dest/classes/mill/define/Worker.tasty
[09486] [error] one error found

Seems to be transient, in that pressing Enter to re-run the specified tasks without any changes makes it go away

[lefou]

I've seen this too. Re-running typically helped.

[lihaoyi]

I'm also seeing ./mill __.compile not picking up breakages sometimes, and I need to hit Enter to re-run the job and have the error messages shown. Not sure if related

[jodersky]

I'm happy to check this out. Do you have any more info to reproduce this (e.g. common situations in which you observe this)?

[lihaoyi]

i don't have any clear repro, but it has something to do with incremental compilation.

Perhaps you could try fuzzing it? Run ./mill -w __.compile and have a script that loops through all the scala files in core, alternately commenting each file out (which will cause a compile error) and restoring it (which should fix the error). Hopefully in the process of all this you'll notice the misbehavior, which is compile failing when a file is restored

[jodersky]

getting back to this

Seems to be transient, in that pressing Enter to re-run the specified tasks without any changes makes it go away

By "pressing Enter", do you mean that this only happens in a watch loop -w? Do you have any other IDEs open that somehow invoke mill (e.g. use BSP)?

I'm asking because I have not run been able to repro this issue with tasty, but sometimes have other spurious errors (like missing classfiles) that appear to happen if I run metals using BSP and also call mill on the command line.

[lihaoyi]

I have IntelliJ open that sometimes invokes Mill through BSP. I've hit the missing classfiles issue too.

I thought we're meant to have a global lock around the Mill output folder, but maybe that somehow isn't working.

If you've managed to find other issues with missing classfiles due to BSP, perhaps the fix is https://github.com/com-lihaoyi/mill/issues/4473. Maybe you could try implementing that and see if the problem goes away?

[lihaoyi]

Unassigning Jakob since he won't have time to get to this

[lihaoyi]

Seems to have mostly gone away recently, maybe with newer scala 3.7.x versions