AvroConvert icon indicating copy to clipboard operation
AvroConvert copied to clipboard

Convert JSON directly to Avro

Open AdrianStrugala opened this issue 4 years ago • 2 comments

Discussed in https://github.com/AdrianStrugala/AvroConvert/discussions/36

Originally posted by vivekkwatra April 7, 2021 Is there any sample to convert JSON string into Avro Generic Record for C#?

FYI similar functionality available in Java https://github.com/allegro/json-avro-converter/blob/master/converter/src/main/java/tech/allegro/schema/json2avro/converter/JsonAvroConverter.java

AdrianStrugala avatar Sep 15 '21 07:09 AdrianStrugala

The goal is introduce new AvroConvert feature:

var avro = AvroConvert.Json2Avro(string json);

Making this method generic, and providing T as type makes the implementation simple. In the case, if the generic parameter is not provided it is hard to determine the parent object.

As example json: {"name":"red","favorite_number":2137,"favorite_color":"blue"}

does not contain information about the parent class, only fields

AdrianStrugala avatar Jun 13 '22 11:06 AdrianStrugala

Most of the work is ready on branch: https://github.com/AdrianStrugala/AvroConvert/tree/feature/Json2Avro.

The only obstacle that is left is: how to distinguish between JSON Dictionary and Object representation. They are both serialized as an array of key-value pairs.

Every key-value pair cannot be directly converted to Avro objects, as they may contain non-string keys (which is not allowed for Avro fields names)

AdrianStrugala avatar Jun 21 '22 12:06 AdrianStrugala

The implementation of Json2Avro feature is done: #84 . There is single exception: Dictionaries are not supported as for dynamic Json2Avro invocation.

AdrianStrugala avatar Dec 13 '22 09:12 AdrianStrugala