BigIntOptionAnyValHolder in AnyValSerializerTest fails in scala 2.11 and scala 3.3

[jtjeferreira]

This test fails in scala 2.11 and scala 3.3 with the follwoing error

com.fasterxml.jackson.databind.JsonMappingException: class com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest$BigIntAnyVal cannot be cast to class java.lang.Number (com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest$BigIntAnyVal is in unnamed module of loader sbt.internal.LayeredClassLoader @7e8791ff; java.lang.Number is in module java.base of loader 'bootstrap') (through reference chain: com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest$BigIntOptionAnyValHolder["value"])
[info]   at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:402)
[info]   at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:361)
[info]   at com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.java:323)
[info]   at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:778)
[info]   at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:184)
[info]   at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:502)
[info]   at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:341)
[info]   at com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4801)
[info]   at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:4042)
[info]   at com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest.$anonfun$2(AnyValSerializerTest.scala:30)
[info]   ...
[info]   Cause: java.lang.ClassCastException: class com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest$BigIntAnyVal cannot be cast to class java.lang.Number (com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest$BigIntAnyVal is in unnamed module of loader sbt.internal.LayeredClassLoader @7e8791ff; java.lang.Number is in module java.base of loader 'bootstrap')
[info]   at com.fasterxml.jackson.databind.ser.std.NumberSerializer.serialize(NumberSerializer.java:23)
[info]   at com.fasterxml.jackson.databind.ser.std.ReferenceTypeSerializer.serialize(ReferenceTypeSerializer.java:389)
[info]   at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:732)
[info]   at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:770)
[info]   at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:184)
[info]   at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:502)
[info]   at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:341)
[info]   at com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4801)
[info]   at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:4042)
[info]   at com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest.$anonfun$2(AnyValSerializerTest.scala:30)

[pjfanning]

Thanks @jtjeferreira I see the failure. I'm afraid that I don't know much about jackson-module-scala AnyVal support but will have a look. There doesn't look like there is any code in jackson-module-scala that handles AnyVals explicitly. So basically, it appears that Jackson is just using the class files and different versions of Scala produce differences in the class files for your use case.

I suspect that we will need to add a new serializer and deserializer that tries to handle AnyVal explicitly and in a way that tries to standardise what happens across all supported Jackson versions. This may not necessarily be easy.

In the end of the day, jackson-module-scala and its reliance on Java Reflection at runtime is always going to struggle to support all Scala use cases across all Scala versions.

[jtjeferreira]

So basically, it appears that Jackson is just using the class files and different versions of Scala produce differences in the class files for your use case.

You are right. If I use javap on the BigIntOptionAnyValHolder, in scala3 the signature of value method is public scala.Option<scala.math.BigInt> value(); vs public scala.Option<com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest$BigIntAnyVal> value(); in scala 2.13

scala 3.3.3

javap target/scala-3.3.3/test-classes/com/fasterxml/jackson/module/scala/ser/AnyValSerializerTest\$BigIntOptionAnyValHolder.class
Compiled from "AnyValSerializerTest.scala"
public class com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest$BigIntOptionAnyValHolder implements scala.Product,java.io.Serializable {
  public static com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest$BigIntOptionAnyValHolder apply(scala.Option<scala.math.BigInt>);
  public static com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest$BigIntOptionAnyValHolder fromProduct(scala.Product);
  public static com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest$BigIntOptionAnyValHolder unapply(com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest$BigIntOptionAnyValHolder);
  public com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest$BigIntOptionAnyValHolder(scala.Option<scala.math.BigInt>);
  public scala.collection.Iterator productIterator();
  public scala.collection.Iterator productElementNames();
  public int hashCode();
  public boolean equals(java.lang.Object);
  public java.lang.String toString();
  public boolean canEqual(java.lang.Object);
  public int productArity();
  public java.lang.String productPrefix();
  public java.lang.Object productElement(int);
  public java.lang.String productElementName(int);
  public scala.Option<scala.math.BigInt> value();
  public com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest$BigIntOptionAnyValHolder copy(scala.Option<scala.math.BigInt>);
  public scala.Option<scala.math.BigInt> copy$default$1();
  public scala.Option<scala.math.BigInt> _1();
}

scala 2.13

$ javap target/scala-2.13/test-classes/com/fasterxml/jackson/module/scala/ser/AnyValSerializerTest\$BigIntOptionAnyValHolder.class
Compiled from "AnyValSerializerTest.scala"
public class com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest$BigIntOptionAnyValHolder implements scala.Product,java.io.Serializable {
  public scala.collection.Iterator<java.lang.String> productElementNames();
  public scala.Option<com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest$BigIntAnyVal> value();
  public com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest$BigIntOptionAnyValHolder copy(scala.Option<com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest$BigIntAnyVal>);
  public scala.Option<com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest$BigIntAnyVal> copy$default$1();
  public java.lang.String productPrefix();
  public int productArity();
  public java.lang.Object productElement(int);
  public scala.collection.Iterator<java.lang.Object> productIterator();
  public boolean canEqual(java.lang.Object);
  public java.lang.String productElementName(int);
  public int hashCode();
  public java.lang.String toString();
  public boolean equals(java.lang.Object);
  public com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest$BigIntOptionAnyValHolder(scala.Option<com.fasterxml.jackson.module.scala.ser.AnyValSerializerTest$BigIntAnyVal>);
}

[jtjeferreira]

I think this ticket might be related...

[jtjeferreira]

I think this https://github.com/scala/scala3/issues/10846 might be related...

https://github.com/scala/scala/pull/8127 was merged in scala 2.12.9. If I run the test with scala 2.12.8 it fails with the ClassCastException. If I run with 2.12.9 it works.

[pjfanning]

@jtjeferreira Thanks for the research. If you are stuck and need a solution that works with existing versions of Jackson - Jackson serialization/deserlization is quite customizable. There are Jackson annotations that you can add to classes and their methods and fields. You could also create a custom serializer and/or deserializer and register it with your mapper.

[jtjeferreira]

this should be fixed in 3.3.4-RC1. I will give it a try in the next days...

[jtjeferreira]

@pjfanning this is fixed for scala3, but still fails for scala 2.11 (as expected). Should I move this test to a different test folder or should we drop scala 2.11 support ;) ?

[pjfanning]

Thanks @jtjeferreira - there will be a Jackson 2.18.0-RC1 release within the next few weeks so I don't want to complicate that by having the Scala 3 build with an RC version of Scala. When Scala 3.3.4 is released, I'll merge this or something similar.

[jtjeferreira]

Thanks @jtjeferreira - there will be a Jackson 2.18.0-RC1 release within the next few weeks so I don't want to complicate that by having the Scala 3 build with an RC version of Scala. When Scala 3.3.4 is released, I'll merge this or something similar.

fair enough. Actually I don't think the upgrade of scala version is necessary in this library, as it should be enough to upgrade application (i.e code that makes use of AnyVal)