avrohugger icon indicating copy to clipboard operation
avrohugger copied to clipboard

SpecificRecord Support for multi record union type

Open xuehaowen opened this issue 6 years ago • 2 comments

I need to deserialize avro message which implemented SpecificRecord. The schema I used is like: union { record1, record2, record3} event;

But I find SpecificRecord does not support OptionShapelessCoproduct, OptionEitherShapelessCoproduct, or OptionalShapelessCoproduct.

val avroScalaCustomTypes = SpecificRecord.defaultTypes.copy(union = OptionEitherShapelessCoproduct) val generator = new Generator(SpecificRecord, restrictedFieldNumber = true, avroScalaCustomTypes = Some(avroScalaCustomTypes))

java.lang.RuntimeException: Unions beyond nullable fields are not supported at scala.sys.package$.error(package.scala:27) at avrohugger.format.specific.converters.JavaConverter$.convertToJava(JavaConverter.scala:35) at avrohugger.format.specific.methods.GetGenerator$.avrohugger$format$specific$methods$GetGenerator$$asGetCase$1(GetGenerator.scala:23)

Can we add support for specific record

xuehaowen avatar Feb 27 '19 18:02 xuehaowen

👍🏼 +1. Same issue.

Would it make sense to support this kind of unions? Because it doesn't work event for two types inside union:

{
  "name": "myField",
  "type": ["MyEvent1", "MyEvent2"]
}

Dr-Nikson avatar Mar 25 '19 19:03 Dr-Nikson

[error] stack trace is suppressed; run last Compile / avroScalaGenerate for the full output
[error] (Compile / managedSources) spray.json.DeserializationException: Object expected in field 'name'
[error] (Compile / avroScalaGenerate) Unions, beyond nullable fields, are not supported. 
[error] Found a union of more than one type

Top level in this avsc file is not a single type like usual, but 4 types are defined inside an array: https://github.com/julianpeeters/avrohugger/issues/24

If the types are independent, it is not an issue to split the file in multiple avsc files.

But if there are dependencies, i do not know currently any way in Avro hugger to ensure that the avsc files are parsed in their dependency order: https://github.com/julianpeeters/avrohugger/issues/137

nicolaemarasoiu avatar Jul 07 '20 13:07 nicolaemarasoiu