avro icon indicating copy to clipboard operation
avro copied to clipboard

[ruby] Keep all union record errors and tag them accordingly

Open davidgm0 opened this issue 2 years ago • 1 comments

I wasn't able to create an issue in JIRA, since I don't seem to be able to request a user. I am a first time contributor, apologies in case something is not precise enough

What is the purpose of the change

Improving validation errors for unions of records When no valid union is found, this line selects the first error that it finds. If there would be 3 different possible schemas, there are 3 failures, however only the first one will be considered.

https://github.com/apache/avro/blob/fc2a4e0e5d88755c776b97fa1872e70ffbe0d4bb/lang/ruby/lib/avro/schema_validator.rb#L199

The returned error also does not give any context of the error, only about the invalid fields.

As an example, consider we have 3 types of adresses PersonalAddress, WorkAddress, SecondAddress and we made a typo: We used the field compan instead of company, which belongs to WorkAddress. No type will work for the union. { "compan" => "acme inc.", "some_other_field" => "something else" }

Since PersonalAddress is the first type that fails, the errors of PersonalAddress are the ones that will be returned. Since PersonalAddress has different structure, all fields that aren't the same as in WorkAddress will be shown as errors. The error will be something like

at .address extra field 'compan' - not in schema
at .address extra field 'some_other_field' - not in schema

This gets very confusing because 1) The issue is that no union matched 2) It is not clear because only one of the error is displayed.

I'd be happy if this is fix in some other fashion, this is just an example. With we replace detect with select and we add the schema name to the error. Then the error will be like:

PersonalAddress at .address extra field 'compan' - not in schema
PersonalAddress at .address extra field 'some_other_field' - not in schema
WorkAddress at .address extra field 'compan' - not in schema
SecondAddress at .address extra field 'compan' - not in schema
SecondAddress at .address extra field 'some_other_field' - not in schema

Verifying this change

This change added tests and can be verified as follows:

  • Modified existing test, existing tests pass

Documentation

  • Does this pull request introduce a new feature? no

davidgm0 avatar Jan 20 '23 15:01 davidgm0

I wasn't able to create an issue in JIRA

JIRA issue is needed because this is what we use for the release changelog later. You can request a JIRA account at [email protected]. Send us your preferred username, display name and email address.

martin-g avatar Jan 23 '23 14:01 martin-g