chill
chill copied to clipboard
Cast error serializing Joda Datetime
Hello, I'm running into an issue serializing org.joda.time.DateTime through Kryo on Java; it appears to fail casting the java model DateTime to long. Does the library support this? Thanks!
In a Spark Kryo registration class:
kryo.register(Foo.class, AvroSerializer.SpecificRecordBinarySerializer(scala.reflect.ClassTag$.MODULE$.apply(Foo.class)));
Where Foo is generated from Avro schema. The field in question is
{"name": "myField", "type": {"type": "long", "logicalType": "timestamp-millis"}},
When serialization is attempted, I end up with this stack:
java.lang.ClassCastException: org.joda.time.DateTime cannot be cast to java.lang.Long
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:79)
at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:114)
at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:104)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)
at com.twitter.bijection.avro.BinaryAvroCodec.apply(AvroCodecs.scala:273)
at com.twitter.bijection.avro.BinaryAvroCodec.apply(AvroCodecs.scala:269)
at com.twitter.chill.InjectiveSerializer.write(InjectiveSerializer.scala:24)
at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628)
at org.apache.spark.serializer.KryoSerializationStream.writeObject(KryoSerializer.scala:195)
...
JFYI, https://github.com/magro/kryo-serializers provides a kryo "native" serializer for joda DateTime.