chill icon indicating copy to clipboard operation
chill copied to clipboard

Cast error serializing Joda Datetime

Open mkhsueh opened this issue 9 years ago • 1 comments

Hello, I'm running into an issue serializing org.joda.time.DateTime through Kryo on Java; it appears to fail casting the java model DateTime to long. Does the library support this? Thanks!

In a Spark Kryo registration class:

 kryo.register(Foo.class, AvroSerializer.SpecificRecordBinarySerializer(scala.reflect.ClassTag$.MODULE$.apply(Foo.class)));

Where Foo is generated from Avro schema. The field in question is

{"name": "myField", "type": {"type": "long", "logicalType": "timestamp-millis"}},

When serialization is attempted, I end up with this stack:

java.lang.ClassCastException: org.joda.time.DateTime cannot be cast to java.lang.Long
    at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:79)
    at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:114)
    at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:104)
    at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
    at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)
    at com.twitter.bijection.avro.BinaryAvroCodec.apply(AvroCodecs.scala:273)
    at com.twitter.bijection.avro.BinaryAvroCodec.apply(AvroCodecs.scala:269)
    at com.twitter.chill.InjectiveSerializer.write(InjectiveSerializer.scala:24)
    at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628)
    at org.apache.spark.serializer.KryoSerializationStream.writeObject(KryoSerializer.scala:195)
...

mkhsueh avatar Aug 24 '16 21:08 mkhsueh

JFYI, https://github.com/magro/kryo-serializers provides a kryo "native" serializer for joda DateTime.

magro avatar Aug 25 '16 07:08 magro