Enum (extends java) => null in java, when scala looks at it before java

[benjhm]

Compiler version

Scala 3.0.0, OpenJDK 15.0.2, sbt 1.5.2,

Minimized code

TestenumS.scala

object TestenumS :
    def go() = println("Scala: Testme Hello= " + Testme.Hello)

enum Testme extends java.lang.Enum[Testme] :
    case Hello

TestenumJ.java

public class TestenumJ {
    public static void main(String[] args) {
    TestenumS.go();    // line 3 calls Scala to look at Testme
    System.out.println("Java: Testme Hello= " + Testme.Hello); //=> null, unless line 3 commented, then => Hello
    }
}

Output

[info] running TestenumJ 
Scala: Testme Hello= Hello
Java: Testme Hello= null

Expectation

Java: Testme Hello= Hello (not null !!)

It works as expected (Hello) only when line 3 is commented, i.e. when java sees the enum before scala. As it compiles ok, so the null propagates to cause runtime errors elsewhere, it can takes a long time to find the cause (especially when it looks ok called from scala)

[benjhm]

Here's another variant of the same problem: enum TimeOfDay works in java, although Greeting doesn't ( => null ) Similar to simpler version above, as TimeOfDay initialises Greeting (in scala code, although viewed in java)

TestenumS.scala

enum Greeting extends java.lang.Enum[Greeting] :
    case Hello
    case Goodbye

import Greeting.*
enum TimeOfDay (val mygreeting : Greeting) extends java.lang.Enum[TimeOfDay] :    
    case Morning extends TimeOfDay (Hello)
    case Afternoon extends TimeOfDay (Goodbye)

TestenumJ.java

public class TestenumJ {
    public static void main(String[] args) {
    System.out.println("Java: TimeOfDay Morning = " + TimeOfDay.Morning); 
    System.out.println("Java: TimeOfDay Morning mygreeting = " + TimeOfDay.Morning.mygreeting()); 
    System.out.println("Java: Greeting Hello = " + Greeting.Hello); 
    }
}

Output:

[info] running TestenumJ 
Java: TimeOfDay Morning = Morning
Java: TimeOfDay Morning mygreeting = Hello
Java: Greeting Hello= null 

[ecartner]

Any word on this? Any chance we can get this mentioned in the Enum docs?

[prolativ]

The problem also occurs if Testme.Hello isn't referenced in scala code but instead in java code one references first Testme$.Hello and then Testme.Hello

[andiogenes]

It seems that initializing classes in a specific order causes null to be stored in an enum field.

---

Here is the bytecode of Testme and Testme$ class initializers produced by Scala 3.0.0:

Testme.<clinit>

  private static {};
    descriptor: ()V
    flags: (0x000a) ACC_PRIVATE, ACC_STATIC
    Code:
      stack=1, locals=0, args_size=0
         0: getstatic     #43                 // Field Testme$.Hello:LTestme;
         3: putstatic     #44                 // Field Hello:LTestme;
         6: return

Testme$.<clinit>

  public static {};
    descriptor: ()V
    flags: (0x0009) ACC_PUBLIC, ACC_STATIC
    Code:
      stack=4, locals=0, args_size=0
         0: new           #2                  // class Testme$
         3: dup
         4: invokespecial #27                 // Method "<init>":()V
         7: putstatic     #29                 // Field MODULE$:LTestme$;
        10: getstatic     #29                 // Field MODULE$:LTestme$;
        13: iconst_0
        14: ldc           #30                 // String Hello
        16: invokespecial #34                 // Method $new:(ILjava/lang/String;)LTestme;
        19: putstatic     #36                 // Field Hello:LTestme;
        22: iconst_1
        23: anewarray     #38                 // class Testme
        26: dup
        27: iconst_0
        28: getstatic     #29                 // Field MODULE$:LTestme$;
        31: pop
        32: getstatic     #36                 // Field Hello:LTestme;
        35: aastore
        36: checkcast     #39                 // class "[LTestme;"
        39: putstatic     #41                 // Field $values:[LTestme;
        42: return

If Testme$ is initialized first (e.g., while being accessed in TestenumS$.go), Testme will be subsequently initialized during the execution of Testme$.$new:

  private Testme $new(int, java.lang.String);
    descriptor: (ILjava/lang/String;)LTestme;
    flags: (0x0002) ACC_PRIVATE
    Code:
      stack=4, locals=3, args_size=3
         0: new           #11                 // class Testme$$anon$1
         3: dup
         4: iload_1
         5: aload_2
         6: invokespecial #85                 // Method Testme$$anon$1."<init>":(ILjava/lang/String;)V
         9: areturn

The execution of 0: new #11 // class Testme$$anon$1 here will lead to the initialization of Testme$$anon$1 (according to JVMS §5.5, "A class or interface C may be initialized only as a result of ... [execution of the instruction] new ... that references C"), which, by itself, inherits Testme:

// Shortened output of `javap Testme\$\$anon\$1.class`
final class Testme$$anon$1 extends Testme implements scala.runtime.EnumValue,scala.deriving.Mirror$Singleton {
  ...
}

Therefore, Testme will be initialized during the initialization of Testme$$anon$1 (read JVMS §5.5, starting from "The procedure for initializing C is then as follows: ...", "7. Next, if C is a class ...").

During the execution of Testme.<clinit>, the static field Testme$.Hello—containing null at the moment—will be read, and this exact value will be assigned to the field Testme.Hello.

---

On the other hand, if Testme is initialized first, execution of getstatic will trigger the initialization of Testme$, including the proper initialization of Testme$.Hello field, and expected output will be observed.

[andiogenes]

It also looks like the same problem occurred in https://github.com/scala/scala3/issues/16391 .

[nau]

Confirm, this still exists in Scala 3.3.6. OpenJDK 64-Bit Server VM Zulu21.36+17-CA (build 21.0.4+7-LTS, mixed mode, sharing). Any workaround for this, guys?

[tanishiking]

I guess a workaround is to use companion object's field (Testme$.Hello) which is consistently initialized correctly, instead of Testme.Hello, FYI @nau.

As @andiogenes explained very clearly, the root cause of this is a static initialization circular dependency between a enum class (Testme) and its companion object (Testme$). This circularity causes the Java-compatible static proxy (e.g., Testme.Hello) to be initialized with null, even though the companion object's field (Testme$.Hello) is initialized correctly.

---

Here are a few potential solutions that come to mind from looking at the bytecode (ignoring binary compatibility and the extent of code changes for now):

• (1) Initialize the proxy field (Testme.Hello) from the companion's initializer:
  • Make Testme$.<clinit> to initialize the static Testme.Hello field after Testme$ has completed its own initialization. This should remove the dependency of Testme.<clinit> on Testme$.Hello.
    • (Instead of the current initialization where Testme.<clinit> copies the value from Testme$.Hello).
  • Drawback: this would require the proxy field (Testme.Hello) to be non-final, which may be undesirable.
• (2) Generate a proxy accessor method:
  • Instead of a proxy field, generate a static method like public static Testme.Hello() { return Testme$.Hello; }.
  • A drawback is, Java programs would need to be modified to call Testme.Hello() instead of accessing the Testme.Hello field. And, the original static field Testme.Hello remain null.
• (3) Do nothing, and document the behavior:
  • Leave the current implementation as-is. Clearly document that when accessing this enum from Java, users must use the companion object's field directly (i.e., Testme$.Hello) instead of the static proxy field (Testme.Hello).

---

scala-cli example and javap result : https://github.com/tanishiking/kitchensink/tree/main/scala3/12637