chill icon indicating copy to clipboard operation
chill copied to clipboard

Avro Deserialization Breaks on Java 17

Open strygul opened this issue 3 years ago • 0 comments

Describe the bug Broken deserialization of Avro schemas in Java 17

To Reproduce


import com.twitter.chill.{KryoPool, ScalaKryoInstantiator}
import org.apache.avro.Schema

object Test extends App {

  private val pool: KryoPool = ScalaKryoInstantiator.defaultPool
  val bytes = pool.toBytesWithClass(Schema.parse("""{
                                       |"type" : "record",
                                       |"namespace" : "Foo",
                                       |"name" : "Employee",
                                       |"fields" : [
                                       |{ "name" : "Name" , "type" : "string" },
                                       |{ "name" : "Age" , "type" : "int" }
                                       |]
                                       |}""".stripMargin))

  val deserialized = pool.fromBytes(bytes)
}

Expected behavior Successful deserialization

Screenshots N/A

Environment Java Amazon Corretto 17.0.4 org.apache.avro 1.8.2 org.twitter.chill 0.10.0

Additional context The code works on Java 8. However, due to the stronger encapsulation rules in the later versions of Java, private inner class constructors (org.apache.avro.Schema$IntSchema in this case) could not be accessed anymore in Java 17 without explicitly making them accessible with the java reflection API:

Exception in thread "main" com.esotericsoftware.kryo.KryoException: Error constructing instance of class: org.apache.avro.Schema$IntSchema
Serialization trace:
schema (org.apache.avro.Schema$Field)
fieldMap (org.apache.avro.Schema$RecordSchema)
	at com.twitter.chill.Instantiators$$anon$1.newInstance(KryoBase.scala:142)
	at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1139)
	at com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldSerializer.java:562)
	at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:538)
	at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
	at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
	at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:543)
	at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:813)
	at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:161)
	at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:39)
	at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
	at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
	at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:543)
	at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:813)
	at com.twitter.chill.SerDeState.readClassAndObject(SerDeState.java:61)
	at com.twitter.chill.KryoPool.fromBytes(KryoPool.java:94)
	at Test$.delayedEndpoint$Test$1(Test.scala:22)
	at Test$delayedInit$body.apply(Test.scala:6)
	at scala.Function0.apply$mcV$sp(Function0.scala:39)
	at scala.Function0.apply$mcV$sp$(Function0.scala:39)
	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
	at scala.App.$anonfun$main$1$adapted(App.scala:80)
	at scala.collection.immutable.List.foreach(List.scala:431)
	at scala.App.main(App.scala:80)
	at scala.App.main$(App.scala:78)
	at Test$.main(Test.scala:6)
	at Test.main(Test.scala)
Caused by: java.lang.IllegalAccessException: class com.twitter.chill.Instantiators$ cannot access a member of class org.apache.avro.Schema$IntSchema with modifiers "public"
	at java.base/jdk.internal.reflect.Reflection.newIllegalAccessException(Reflection.java:392)
	at java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:674)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:489)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
	at com.twitter.chill.Instantiators$.$anonfun$normalJava$1(KryoBase.scala:174)
	at com.twitter.chill.Instantiators$$anon$1.newInstance(KryoBase.scala:139)
	... 26 more

It has to do with the call to com.esotericsoftware.reflectasm that utilizes the following code:

val access = ConstructorAccess.get(Class.forName("org.apache.avro.Schema$StringSchema"))
 access.newInstance

strygul avatar Jul 22 '22 07:07 strygul