erlavro
erlavro copied to clipboard
backwards and forward schema evolution
I was trying out erlavro for schema evolution. I'm not sure what the stance on evolution is here. Is it not supported? Or is there a way to support it that I'm not aware of. For example:
the schema (block-ethereum.avsc
) has a lot of fields deleted (in contrast to the data provided which was encoded using an earlier version). No additional fields are added or modification done on any of the retained fields in this new schema:
{
"type":"record",
"namespace":"com.covalenthq.brp.avro",
"name":"ReplicationSegment",
"fields":[
{
"name":"startBlock",
"type":"long"
},
{
"name":"endBlock",
"type":"long"
},
{
"name":"elements",
"type":"long"
},
{
"name":"codecVersion",
"type":"double",
"default":0.33
}
]
}
the encoded binary is here
when i try running the decoder:
iex> {:ok, scheman} = Avrora.Schema.Name.parse("block-ethereum")
iex> scheman1 = %Avrora.Schema{full_name: scheman.name}
iex> {:ok, schema} = Avrora.Resolver.resolve(scheman1.full_name)
iex> :avro_binary_decoder.decode(specimen, schema.full_name, schema.lookup_table, Avrora.AvroDecoderOptions.options())
** (MatchError) no match of right hand side value: {%{"codecVersion" => 4.3256427312130535e-37, "elements" => 24, "endBlock" => 66, "startBlock" => 1}, <<97, 100, 54, 99, 57, 57, 54, 51, 97, 56, 55, 52, 56, 48, 54, 97, 51, 102, 50, 51, 51, 49, 49, 50, 50, 99, 56, 50, 99, 54, 50, 55, 48, 49, 98, 55, 54, 48, 55, 50, 98, 48, 100, 55, 55, 50, 49, 56, ...>>}
(erlavro 2.9.8) /Users/sudeep/repos/rudder/deps/erlavro/src/avro_binary_decoder.erl:66: :avro_binary_decoder.decode/4
iex:6: (file)
How can i support backwards of forward evolution of schema here?