Revise existing case classes to prepare a better binary compatibility story

[lefou]

Following this document: Binary Compatibility for library authors, we should revise our existing API to prepare easier backward compatible changes.

For case classes, this means:

• make the primary constructor private (this makes the copy method of the class private as well)
• define a private unapply function in the companion object (note that by doing that the case class loses the ability to be used as an extractor in match expressions)
• create a public constructor by defining an apply method in the companion object (it can use the private constructor)

Optional, if the case class is meant to be changed by the API use regularly, you should also add a withXXX-method. Otherwise, the apply-method should be already sufficient.

• for all the fields, define withXXX methods on the case class that create a new instance with the respective field changed (you can use the private copy method to implement them)

[lihaoyi]

I think once the @unroll SIP lands in Scala 3, we should be able to use case classes directly without needing these workarounds, and @unroll new fields to maintain compatibility. We'll need to verify that it works, but if it does it should remove a lot of the manual work necessary to keep our case classes backwards compatible

[lefou]

Ok, that must be new. I didn't know @unroll will also handle copy and unapply.

[lihaoyi]

Yeah it handles new, copy, apply, and unapply (the last one only in Scala 3). It basically makes adding fields "free" w.r.t. binary compatibility as long as (a) they are added on the right and (b) they have default values. So we can hopefully stop all the annoying forwarders and workarounds and just use vanilla case classes .

[lefou]

So, how do I prevent an unapply to appear in a case class in the first place. Do I have to add a @unroll in the first version already?

[lihaoyi]

In Scala 3 unapply is defined as def unapply(value: Foo): Foo, so there isn't any binary compatibility concern with the unapply method when adding a case class, unlike in Scala 2 where it returned an Option[TupleN[...]]. The signature doesn't change when you add fields, and so bytecode compiled against an older version of the case class should continue to work as long as the new version has a superset of the old version's fields

[lihaoyi]

Since @unroll is still experimental in Scala 3.7, and we won't be able to bump past 3.7 until the next breaking version, it means we will likely need to fall back to http://github.com/com-lihaoyi/unroll to provide @unroll functionality in the next major version of Mill. Not a blocker, just something to be aware of

[lefou]

So, does that mean we don't need to take any precaution regarding case classes when rolling out Mill 1.0.0? I understand @unroll will just generate all the overloads to keep binary compatibility, in case we need it. But how about a case class being used is a pattern match? That typically broke, once we changed the case class, hence we made the unapply private.

[lefou]

Oh I see, the default values make the magic.

[lihaoyi]

With https://github.com/com-lihaoyi/mill/pull/5526 we now make use of @com.lihaoyi.unroll