Compile times in Scala 3 are very slow since 0.11.4-M2

[lihaoyi]

So much so that we had to roll back the OS-Lib version when upgrading Mill from Scala 2->3 in https://github.com/com-lihaoyi/mill/pull/3369. We were seeing maybe 20x compile slowdowns, with 15s compiles instead taking 300s.

Most likely culprit is the macro changes introduced in https://github.com/com-lihaoyi/os-lib/pull/329. Probably some naive implementations need optimizing.

CC @pawelsadlo in case anything springs to mind, otherwise I'll get around to looking into it eventually

[pawelsadlo]

@lihaoyi yeah it might be

private def reduceUps(in: Array[String]): List[String] =
    in.foldLeft(List.empty[String]) { case (acc, x) =>
      acc match {
        case h :: t if h == ".." => x :: acc
        case h :: t if x == ".." => t
        case _ => x :: acc
      }
    }.reverse

Probably prepending elements to list with :: has biggest impact on compilation time, but to get best optimization we should probably go for some mutable data structure.

[pawelsadlo]

I will get it done as soon as I have time, however if anyone wants to pick ,just feel free

[lihaoyi]

That code looks very reasonable to me. :: prepending to a list is meant to be fast? But maybe there's something unusual going on that I'm not seeing here

[lihaoyi]

I saw some other compile performance issues that were fixed going from scala 3.5.0 to 3.5.2. Let me see if this one follows suit or whether it sticks around

[pawelsadlo]

Back in the days of early scala 3, prepending to tuple with *: was raising compilation time a lot, that's why I suspect :: . Maybe it's just an object creation that is costly, and we do it every :: call.

[pawelsadlo]

@lihaoyi I have done some benchmarking on minimised literal syntax usage, and didn't notice any big difference in compilation time between M1 and M2 os-lib milestone.

based on referenced PR I assume that it occured while compiling mill itself(?).

Did you check if slowdown is present only on M2 (not on M1)?

[lihaoyi]

I think I did, let me go back and bisect properly to try and verify