jackson-dataformats-binary icon indicating copy to clipboard operation
jackson-dataformats-binary copied to clipboard

Add support for serializing Avros' `GenericRecord` etc types

Open cowtowncoder opened this issue 8 years ago • 4 comments

While maybe not the biggest problem in the world, there may be use cases where it'd be nice to let users essentially convert data bound in Avro-specific containers (like GenericContainer) into more sensible types (like basic Maps and Collections, say; or just POJOs). One way to achieve this would be to add custom serializers for said types.

Another related feature would be the reverse (construct these types from streaming input, i.e add JsonSerializers); that can be a follow-up step if need be.

cowtowncoder avatar May 06 '16 02:05 cowtowncoder

I think this would be really handy. I've written some code to go from Jackson annotated POJOs to Parquet via Avro and it seems that the only way to achieve this with Jackson is:

Convert Pojos to Avro (byte array) -> Convert byte array to GenericRecord -> Convert GenericRecord to Parquet

Having a direct conversion from POJO to Avro GenericRecord and the like would simplify things greatly, and I assume improve performance.

HunterSherms avatar Aug 29 '17 14:08 HunterSherms

Yes, this would be useful. The only (?) tactical question here is whether it'd make sense to add maybe jackson-datatype-avro, since we may want to avoid dep to std avro lib in future (currently we still use it for serialization, schema handling) -- may or may not happen. Conceptually this would then be a datatype, and would nicely plug in via ser/deser interfaces. At the same time of course there's the question of practicality: just adding support here would be simpler. Maybe this package could plug in datatype handlers optionally, via AvroMapper (and allow use as separate thing too).

cowtowncoder avatar Aug 29 '17 21:08 cowtowncoder

I second the need, with the same use case as @HunterSherms.

By the way @HunterSherms are you still using this solution or have you found better ways to avoid extra conversion?

victornoel avatar Apr 12 '19 07:04 victornoel

Unfortunately I did not find a better solution. After writing the code linked above I moved on to other projects. Best of luck!

HunterSherms avatar Apr 12 '19 14:04 HunterSherms