Derivation fails under 3.1.2

[Lasering]

Compiler version

3.1.2

Minimized code

Scastie Link

import scala.deriving.Mirror
import scala.compiletime.{erasedValue, summonFrom}

inline def summonEncoders[T <: Tuple]: List[Encoder[?]] =
  inline erasedValue[T] match
    case _: EmptyTuple => Nil
    case _: (t *: ts) => summonEncoder[t] :: summonEncoders[ts]

inline def summonEncoder[A]: Encoder[A] =
  summonFrom {
    case encodeA: Encoder[A] => encodeA
    case _: Mirror.Of[A] => Encoder.derived[A]
  }

trait Encoder[A]

object Encoder:
  given listEncoder[A: Encoder]: Encoder[List[A]] = new Encoder[List[A]] {}
  
  inline final def derived[A](using mirror: Mirror.Of[A]): Encoder[A] =
    val elemEncoders = summonEncoders[mirror.MirroredElemTypes]
    new Encoder[A] {}

object Component:
  given encoder[Q]: Encoder[Component[Q]] = new Encoder[Component[Q]] {}
case class Component[Q](quantity: Q)

case class Element(priceComponents: List[Component[?]]) derives Encoder

Output

This is the output under 3.1.3-RC2 because the error is better explained (shows inline stack trace). But under 3.1.2 the error is the same.

[error] -- [E007] Type Mismatch Error: /home/simon/scala-seed-project/src/main/scala/example/Hello.scala:30:64 
[error] 30 |case class Element(priceComponents: List[Component[?]]) derives Encoder
[error]    |                                                                ^
[error]    |                       Found:    example.Encoder[example.Component[Any]]
[error]    |                       Required: example.Encoder[example.Component[?]]
[error]    |----------------------------------------------------------------------------
[error]    |Inline stack trace
[error]    |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[error]    |This location contains code that was inlined from Hello.scala:13
[error] 13 |    case encodeA: Encoder[A] => encodeA
[error]    |                  ^
[error]    |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[error]    |This location contains code that was inlined from Hello.scala:13
[error]  9 |    case _: (t *: ts) => summonEncoder[t] :: summonEncoders[ts]
[error]    |                         ^^^^^^^^^^^^^^^^
[error]    |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[error]    |This location contains code that was inlined from Hello.scala:13
[error] 23 |    val elemEncoders = summonEncoders[mirror.MirroredElemTypes]
[error]    |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[error]     ----------------------------------------------------------------------------
[error]    |----------------------------------------------------------------------------
[error]    | Explanation (enabled by `-explain`)
[error]    |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[error]    |
[error]    | Tree: example.Component.encoder[Any]
[error]    | I tried to show that
[error]    |   example.Encoder[example.Component[Any]]
[error]    | conforms to
[error]    |   example.Encoder[example.Component[?]]
[error]    | but the comparison trace ended with `false`:
[error]    |
[error]    |   ==> example.Encoder[example.Component[Any]]  <:  example.Encoder[example.Component[?]]
[error]    |     ==> example.Component[?]  <:  example.Component[Any]
[error]    |       ==> Any  <:  example.Component[?]#Q
[error]    |       <== Any  <:  example.Component[?]#Q = false
[error]    |     <== example.Component[?]  <:  example.Component[Any] = false
[error]    |   <== example.Encoder[example.Component[Any]]  <:  example.Encoder[example.Component[?]] = false
[error]    |
[error]    | The tests were made under the empty constraint
[error]     ----------------------------------------------------------------------------

Expectation

The code should compile under 3.1.2 since it compiles under 3.1.1.

Remarks

1. Under 3.1.1, 3.1.2 and 3.1.3-RC2 val c produces Cannot prove that Encoder[Component[Any]] <:< Encoder[Component[?]]. Since val a and val b compile fine this seems weird.

val a = summon[Encoder[Any] <:< Encoder[?]]
val b = summon[Component[Any] <:< Component[?]]
val c = summon[Encoder[Component[Any]] <:< Encoder[Component[?]]]

2. Removing the context bound in listEncoder makes the code compile.
3. Removing the line val elemEncoders makes the code to compile. This code is a minimization, under the real use case elemEncoders is used to actually drive the derivation. See this for the full code.
4. Removing the summonEncoder method and changing summonEncoders to the following makes the code compile:

inline def summonEncoders[T <: Tuple]: List[Encoder[?]] =
  inline erasedValue[T] match
    case _: EmptyTuple => Nil
    case _: (t *: ts) =>
      summonFrom {
        case encodeA: Encoder[t] => encodeA
        case _: Mirror.Of[t] => Encoder.derived[t]
      } :: summonEncoders[ts]

[nicolasstucki]

Minimized to

trait Encoder[T]
trait Component[T]

inline def summonEncoder[A]: Encoder[A] =
  scala.compiletime.summonFrom {
    case encodeA: Encoder[A] => encodeA
  }

given listEncoder[A: Encoder]: Encoder[List[A]] = ???
given componentEncoder[A]: Encoder[Component[A]] = ???

def test: Unit = summonEncoder[List[Component[?]]]

-- [E007] Type Mismatch Error: Foo.scala:12:30 ---------------------------------
12 |def test: Unit = summonEncoder[List[Component[?]]]
   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |                 Found:    Encoder[Component[Any]]
   |                 Required: Encoder[Component[?]]
   |----------------------------------------------------------------------------
   |Inline stack trace
   |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   |This location contains code that was inlined from Foo.scala:6
 6 |    case encodeA: Encoder[A] => encodeA
   |                  ^
    ----------------------------------------------------------------------------
   |
   | longer explanation available when compiling with `-explain`

[nicolasstucki]

The issue started in 8ed6bde18cf76c423cd98979643b6d5ef0ab1a81. @smarter is this an intended change?

[Lasering]

While this isn't solved, does anyone have a suggestion on how to work around it?

[SethTisue]

Under 3.1.1, 3.1.2 and 3.1.3-RC2 val c produces Cannot prove that Encoder[Component[Any]] <:< Encoder[Component[?]] Since val an and val b compile fine this seems weird. val a = summon[Encoder[Any] <:< Encoder[?]] val b = summon[Component[Any] <:< Component[?]] val c = summon[Encoder[Component[Any]] <:< Encoder[Component[?]]]

This is something of an FAQ in Scala, even back in Scala 2 days.

Encoder[Component[?]] doesn't mean Encoder[Component[T]] forSome { type T }, as you apparently expected. Rather, it means something fundamentally different, namely Encoder[Component[T] forSome { type T}], and that's why the val c line doesn't compile.

[SethTisue]

I think we need to consider the possibility that 3.1.2 is right here and it's 3.1.1 that was wrong. I'm not saying I can prove it (yet?). I'm just saying, we cannot take 3.1.1's behavior as gospel. We must actually analyze and understand what's correct.

Note that the presence of Any tends to lead one down a mental garden path where one wants to use variance to justify things, but variance isn't in play here. Encoder and Component are both invariant.

"type Any" != "any type"

[s5bug]

Just looking through the type trace:

[error]    |   ==> Encoder[Component[Any]]  <:  Encoder[Component[?]]
[error]    |     ==> Component[?]  <:  Component[Any]
[error]    |       ==> Any  <:  Component[?]#Q
[error]    |       <== Any  <:  Component[?]#Q = false
[error]    |     <== Component[?]  <:  Component[Any] = false
[error]    |   <== Encoder[Component[Any]]  <:  Encoder[Component[?]] = false

1. Encoder is invariant, so to verify that Encoder[? >: L1 <: U1] <: Encoder[? >: L2 <: U2], we have to verify L2 <: L1 <: U1 <: U2.
2. Component is invariant, so to verify that Component[? >: L1 <: U1] <: Component[? >: L2 <: U2], we have to verify L2 <: L1 <: U1 <: U2.
3. Encoder[Component[Any]] is equivalent to Encoder[? >: Component[Any] <: Component[Any]].
4. Encoder[Component[?]] is equivalent to Encoder[? >: Component[?] <: Component[?]].
5. Component[Any] is equivalent to Component[? >: Any <: Any].
6. Component[?] is equivalent to Component[? >: Nothing <: Any].
7. (1, 3, 4) Encoder[Component[Any]] <: Encoder[Component[?]] requires Component[?] <: Component[Any] <: Component[Any] <: Component[?].
8. (2, 5, 6) Component[?] <: Component[Any] requires Any <: Nothing <: Any <: Any.
9. Any <: Nothing is false.

More generally, it is as Seth describes: a Component[?] (a component of any type) is not a Component[Any]. Consider a more complex Component that provides a function render: Q => String: Component[?]#render is never callable, because you don't know what type it has, whereas Component[Any]#render is always callable.

I speculate that the reason this ends up not working is because the Component encoder takes a specific type parameter Q, rather than allowing for an arbitrarily bounded wildcard. Consider that

trait Encoder[Q] {}

trait Component[Q]

object Component {
  given[Q]: Encoder[Component[Q]] with {}
}

summon[Encoder[Component[?]]]

throws a compilation error, while

trait Encoder[Q] {}

trait Component[Q]

object Component {
  given[L <: U, U]: Encoder[Component[? >: L <: U]] with {}
}

summon[Encoder[Component[?]]]

does not.

If we make this change change @Lasering's Scastie, it seems everything compiles with no issue:

Scastie link

import scala.deriving.Mirror
import scala.compiletime.{erasedValue, summonFrom}

inline def summonEncoders[T <: Tuple]: List[Encoder[?]] =
  inline erasedValue[T] match
    case _: EmptyTuple => Nil
    case _: (t *: ts) => summonEncoder[t] :: summonEncoders[ts]

inline def summonEncoder[A]: Encoder[A] =
  summonFrom {
    case encodeA: Encoder[A] => encodeA
    case _: Mirror.Of[A] => Encoder.derived[A]
  }

trait Encoder[A]

object Encoder:
  given listEncoder[A: Encoder]: Encoder[List[A]] = new Encoder[List[A]] {}
  
  inline final def derived[A](using mirror: Mirror.Of[A]): Encoder[A] =
    val elemEncoders = summonEncoders[mirror.MirroredElemTypes]
    new Encoder[A] {}

object Component:
  given encoder[L <: U, U]: Encoder[Component[? >: L <: U]] = new Encoder[Component[? >: L <: U]] {}
case class Component[Q](quantity: Q)

case class Element(priceComponents: List[Component[?]]) derives Encoder

[s5bug]

Even explicitly, a list of Encoders shouldn't really be able to be generated with the old signature:

def encoders(l: List[Component[?]]): List[Encoder[Component[?]]] = l.map {
  case _: Component[t] =>
    val newEncoder: Encoder[Component[t]] = Component.encoder[t]
    (newEncoder: Encoder[Component[?]]) // Error: Encoder is invariant, and Component[?] <!: Component[t]
}

I do wonder what exact calls were made by derivation in 3.1.1 for the original code to work. Is there any way we can look at a reified/annotated source for the derived encoder?

[dwijnand]

[..] while

trait Encoder[Q] {}

trait Component[Q]

object Component {
  given[L <: U, U]: Encoder[Component[? >: L <: U]] with {}
}

summon[Encoder[Component[?]]]

does not.

Even simpler:

  given encoder: Encoder[Component[?]] = new Encoder[Component[?]] {}

Seeing as encoder is universally quantified on Q and that type isn't used in any other way, might as well (assuming no mutable state) have it be a single instance that can handle any component.