linking error to java's Consumer.accept when using Scala.js

[jv3x]

In scala3/js (scala 3.2.0, scalajs 1.11.0), but not in scala2/js, there is linking error "Referring to non-existent method ....accept(java.lang.Object)void" in the following:

  def foo(): Unit = {
    val set = new util.LinkedHashSet[String]
    set.forEach(bar)
  }

  def bar(v: String): Unit = ()

I can workaround it like this in foo:

  val barc: util.function.Consumer[String] = bar
  set.forEach(barc)

There is "(modName / Compile / fastLinkJS) There were linking errors". Couldn't reproduce on the JVM.

Sorry if reporting to wrong repo, scalajs says if it is only on 3.x, we should report here.

[sjrd]

I can reproduce on the JVM with the following minimization:

trait MyConsumer[T] {
  var x: Int = 1
  def accept(x: T): Unit
}

object Test {
  def main(args: Array[String]): Unit = {
    val c: MyConsumer[_ >: String] = x => ()
    c.accept("foo")
  }
}

Mandatory ingredients:

• A Scala-level SAM that is not a JVM SAM (or a platform SAM in the general case) -> removing the var x: Int = 1 definition makes the test pass.
• The wildcard argument in MyConsumer[_ >: String] -> using MyConsumer[String] instead makes the test pass

The workaround from the original post works because it uses an expected type of Consumer[String], not Consumer[_ >: String].

---

Showing the trees before and after erasure definitely paints erasure as the culprit:

[[syntax trees at end of MegaPhase{pruneErasedDefs, uninitialized, inlinePatterns, vcInlineMethods, seqLiterals, intercepted, getters, specializeFunctions, specializeTuples, liftTry, collectNullableFields, elimOuterSelect, resolveSuper, functionXXLForwarders, paramForwarding, genericTuples, letOverApply, arrayConstructors}]] // tests\run\hello.scala
package <empty> {
  @SourceFile("tests/run/hello.scala") trait MyConsumer[T]() extends Object {
    private type T
    def x: Int = 1
    def x_=(x$1: Int): Unit = ()
    def accept(x: T): Unit
  }
  final lazy module val Test: Test = new Test()
  @SourceFile("tests/run/hello.scala") final module class Test() extends Object(
    )
   {
    private def writeReplace(): AnyRef =
      new scala.runtime.ModuleSerializationProxy(classOf[Test.type])
    def main(args: Array[String]): Unit =
      {
        val c: MyConsumer[? >: String] =
          {
            def $anonfun(x: String): Unit = ()
            {
              final class $anon() extends Object(), MyConsumer[? >: String] {
                final def accept(x: String): Unit = $anonfun(x)
              }
              new Object with MyConsumer[? >: String] {...}()
            }
          }
        c.accept("foo")
      }
  }
}

[[syntax trees at end of                   erasure]] // tests\run\hello.scala
package <empty> {
  @SourceFile("tests/run/hello.scala") trait MyConsumer() extends Object {
    def x(): Int = 1
    def x_=(x$1: Int): Unit = ()
    def accept(x: Object): Unit
  }
  final lazy module val Test: Test = new Test()
  @SourceFile("tests/run/hello.scala") final module class Test() extends Object(
    )
   {
    private def writeReplace(): Object =
      new scala.runtime.ModuleSerializationProxy(classOf[Test])
    def main(args: String[]): Unit =
      {
        val c: MyConsumer =
          {
            def $anonfun(x: String): Unit = ()
            {
              final class $anon() extends Object(), MyConsumer {
                final def accept(x: String): Unit = $anonfun(x)
              }
              new Object with MyConsumer {...}():MyConsumer
            }
          }
        c.accept("foo")
      }
  }
}

Removing the _ >: causes erasure to correctly add a bridge for accept(x: Object): Unit.

---

So this is not a Scala.js issue, but an erasure issue.

[sjrd]

Investigating further, it's not Erasure's fault either. It's either Typer or ExpandSAMs, depending on what are the invariants we want. After ExpandSAMs, the tree for the closure is:

        val c: MyConsumer[? >: String] =
          {
            def $anonfun(x: String): Unit = println(x)
            {
              final class $anon() extends Object(), MyConsumer[? >: String] {
                final def accept(x: String): Unit = $anonfun(x)
              }
              new Object with MyConsumer[? >: String] {...}()
            }
          }

which is wrong: it has MyConsumer[? >: String] in the extends clause, but accept takes a String. In fact, if we write this by hand, RefChecks complains:

-- Error: tests\run\i16065.scala:10:37 -----------------------------------------
10 |    val c: MyConsumer[_ >: String] = new MyConsumer[_ >: String] {
   |                                     ^
   |object creation impossible, since def accept(x: T): Unit in trait MyConsumer is not defined
   |(Note that
   | parameter T in def accept(x: T): Unit in trait MyConsumer does not match
   | parameter String in def accept(x: String): Unit in anonymous class Object with MyConsumer[? >: String] {...}
   | )

So why does ExpandSAMs generate that? Well because already after Typer, the tree is dubious:

        val c: MyConsumer[? >: String <: Any] =
          {
            def $anonfun(x: String): Unit = println(x)
            closure($anonfun:MyConsumer[? >: String])
          }

The closure node already mentions MyConsumer[? >: String], but points to $anonfun(String).

I suspect this portion of the typer: https://github.com/lampepfl/dotty/blob/c3ba2f4bf137e6834f081aaf9927e1ffe9a48e13/compiler/src/dotty/tools/dotc/typer/Typer.scala#L1517-L1531 In particular, note that with isWildcardClassSAM it rejects a ? if MyConsumer is a class, but explicitly allows a trait. Sure enough, if we define MyConsumer as a class, we get:

-- Error: tests\run\i16065.scala:8:52 ------------------------------------------
8 |    val c: MyConsumer[_ >: String] = x => println(x)
  |                                                    ^
  |result type of lambda is an underspecified SAM type MyConsumer[? >: String]

Allowing wildcards for traits was most likely done to support Java-style closures, which use use-site variance everywhere. And rejecting classes was most likely done as a proxy to detecting things that would not be LambdaMetaFactory-capable. For LMF stuff, it "works out", because bridges are taken care of by the backend, who doesn't care about wildcards. But clearly, that proxy is not good enough, since it allows the MyConsumer in my repro.

We might think that we should reject non-LMF-capable things there instead of just classes, but that would mean that what SAMs are actually language-level SAMs becomes platform-dependent, and that's very, very bad.

So the solution is not to reject classes, and instead correctly get rid of such wildcards by choosing the appropriate bound. Upper bound if it is used in covariant position in the SAM type, or Lower bound in contravariant. If used in both, the code must be rejected.

---

At this point, this issue exceeds my area of expertise, so I will throw it back to someone typer-savvy. To spare you the trouble, here are test cases:

// i16065.scala

abstract class MyClassConsumer[T] {
  def accept(x: T): Unit
}

trait MyTraitConsumer[T] {
  var x: Int = 1
  def accept(x: T): Unit
}

abstract class MyClassProducer[T] {
  def produce(): T
}

trait MyTraitProducer[T] {
  var x: Int = 1
  def produce(): T
}

object Test {
  def main(args: Array[String]): Unit = {
    val c1: MyClassConsumer[_ >: String] = x => println(x)
    c1.accept("MyClassConsumer")

    val c2: MyTraitConsumer[_ >: String] = x => println(x)
    c2.accept("MyTraitConsumer")

    val p1: MyClassProducer[_ <: String] = () => "MyClassProducer"
    println(p1.produce())

    val p2: MyTraitProducer[_ <: String] = () => "MyTraitProducer"
    println(p2.produce())
  }
}

// i16065.check
MyClassConsumer
MyTraitConsumer
MyClassProducer
MyTraitProducer

[smarter]

Thanks for the minimization and investigation, another piece of the puzzle here is the SAMType extractor which is where we strip the wildcards from the method type we use to represent the closure: https://github.com/lampepfl/dotty/blob/c3ba2f4bf137e6834f081aaf9927e1ffe9a48e13/compiler/src/dotty/tools/dotc/core/Types.scala#L5414-L5430 Either we should instead strip wildcards from Closure#tpt (so generate closure($anonfun:MyConsumer[String]) in your example) or ExpandSAM should strip wildcards itself (because wildcards are not supposed to be allowed in the extends clause)

[odersky]

Is it OK to assign it to you @smarter?

[smarter]

Sure.

[sjrd]

This came back again as #16388.