console icon indicating copy to clipboard operation
console copied to clipboard

Support for Avro logicalType

Open voidcontext opened this issue 4 years ago • 3 comments

When an avro encoded message is displayed as JSON, then attributes that have a logicalType are not rendered properly. In my specific case, the attribute's schema looks like this:

...,
{
  "name": "someAttr",
  "type": {
    "type": "bytes",
    "logicalType":"decimal",
    "precision":8,
     "scale":2
  }
}
...

In the UI it seems like the byte value is getting converted to a string instead of the value being displayed as a decimal value.

Would it be possible to display the value based on the logicalType instead of the type?

voidcontext avatar Jul 30 '21 09:07 voidcontext

Hey Gabor, I wasn't aware that there's a special meaning of the type attribute for Avro encoded messages. Currently we just try to convert the messages to JSON and use on the open source JSON viewers to view the message. Therefore I'm afraid that this is not easily possible, but we can keep this open in case there will be an option in the future.

weeco avatar Aug 02 '21 14:08 weeco

Can you post an example please? Screenshot would be helpful too 😄

rikimaru0345 avatar Sep 06 '21 19:09 rikimaru0345

I'm having a similar issue, this is what it looks like in UI:

Screen Shot 2022-08-08 at 9 16 17 PM

The schema I'm using:

{
	"type": "bytes",
	"scale": 18,
	"precision": 100,
	"logicalType": "decimal"
}

And just to clarify, this is a part of Avro's spec: https://avro.apache.org/docs/1.11.1/specification/#decimal.

It looks like the latest version of goavro supports logical types, so something is missing in the deserialization logic...

sap1ens avatar Aug 09 '22 04:08 sap1ens

Hey @voidcontext and @sap1ens , could you check whether this is fixed with the latest Console version?

Docker image: vectorized/console-unstable:master-2c6f266

weeco avatar Nov 22 '22 21:11 weeco

@weeco yep, it works! Thanks 👍

sap1ens avatar Nov 22 '22 22:11 sap1ens