avromatic icon indicating copy to clipboard operation
avromatic copied to clipboard

#avro_message_value should always return binary encoded string

Open leehambley opened this issue 5 years ago • 0 comments

This almost certainly depends on environmental factors, however in a Rails app where we have a fairly vanilla set-up with Avromatic calling #avro_message_value yields an UTF-8 encoded string.

That's pretty harmless, except that when comparing strings (for e.g in tests) which this project does to guard against other teams sneakily changing Avro schemas the UTF-8 has pretty variable encoding e.g:

[0, 0, 0, 0, 0, 128, 248, 192, 198, 221, 87, 40, 130, 218, 196, 9].pack("c*").force_encoding(Encoding::UTF_8)
"\u0000\u0000\u0000\u0000\u0000\x80\xF8\xC0\xC6\xDDW(\x82\xDA\xC4\t"

[0, 0, 0, 0, 0, 128, 248, 192, 198, 221, 87, 40, 130, 218, 196, 9].pack("c*").force_encoding(Encoding::ASCII_8BIT)
"\x00\x00\x00\x00\x00\x80\xF8\xC0\xC6\xDDW(\x82\xDA\xC4\t"

It should be as simple as forcing the encoding on that method return value before returning it.

The value for new strings can (I believe) depend on Encoding.default_internal and .default_external which can be controlled by env vars and other systemwide configuration.

leehambley avatar Jul 18 '19 11:07 leehambley