unicode characters displayed in message content with schema registry

[nitin1677]

Describe the bug

Topic with schema registry, message content is displaying unicode characters instead of actual numbers for decimal data types or BigAmount fields.

Set up

docker run -> provectuslabs/kafka-ui:latest -> vv0.4.0 (521ba0c)

Steps to Reproduce

Steps to reproduce the behavior:

Schema Registry: { "name": "forexBaseRate", "type": [ "null", { "type": "bytes", "logicalType": "decimal", "precision": 24, "scale": 16 } ], "doc": "forex currency base rate.", "default": null }, { "name": "unitPriceBase", "type": [ "null", { "type": "record", "name": "BigAmount_30_10", "namespace": "com.dev.avro.schema.common", "fields": [ { "name": "amount", "type": { "type": "bytes", "logicalType": "decimal", "precision": 30, "scale": 10 } }, { "name": "currency", "type": { "type": "string", "avro.java.string": "String" } } ] } ], "doc": "Unit Price Base amount.", "default": null },

Expected behavior

Expected: "forexBaseRate": 1.2094000000000000, "unitPriceSystem": { "amount": 1.2094000000, "currency": "USD" },

Unicode values instead of actual numbers in kafka-ui messages content: "forexBaseRate": { "bytes": "#†òoÁ\u0000\u0000" }, "unitPriceBase": { "com.dev.avro.schema.common.BigAmount_30_10": { "amount": "=Å\u001c½\u0000", "currency": "USD" } } Screenshots

n/a

Additional context

n/a

[Haarolean]

Hey, thanks for reaching out. I have a good feeling that it's gonna get fixed within #1777. I'll keep this open for now.

[sirzeta]

I tested 049b35fc99fd09d270fbccb3f0a2bbcc9bf11f56 (049b35f) deployed locally with docker.

Same problem with logical types like decimal with type bytes, no problems with other logical types with other types than bytes :

{
  "type": "record",
  "name": "JournalEntry",
  "namespace": "bookkeeping.cmd.journal.entry",
  "fields": [
    {
      "name": "referenceId",
      "type": {
        "type": "string",
        "logicalType": "uuid"
      }
    },
    {
      "name": "postDate",
      "type": {
        "type": "long",
        "logicalType": "timestamp-millis"
      }
    },
    {
      "name": "journalLines",
      "type": {
        "type": "array",
        "items": {
          "type": "record",
          "name": "JournalLine",
          "fields": [
            {
              "name": "amount",
              "type": {
                "type": "bytes",
                "logicalType": "decimal",
                "precision": 20,
                "scale": 4
              }
            }
          ]
        }
      }
    }
  ]
}

Result:

{
  "referenceId": "049f2831-0635-4abe-a7b0-da534092bc42",
  "postDate": 1609470000000,
  "journalLines": [
    {
      "amount": "\u0000\u0000\u0000\u0000\u00011-\u0000"
    },
    {
      "amount": "\u0000\u0000\u0000\u0000\u0000[€"
    },
    {
      "amount": "\u0000\u0000\u0000\u0000\u0000՟€"
    }
  ]
}

I'm developing a node.js app with @ovotech/avro-decimal to encode/decode from the schema registry, works flawlessly, just at the backend side I decode the bytearray value to string and passing it to frontend and transform again to decimal with decimal.js

Maybe the bytes type handling is the problem? Avro Decimal spec

[Haarolean]

@nitin1677 @sirzeta can you check the master-labeled docker image? Does selecting other serdes to view/produce messages help?

[github-actions[bot]]

This issue has been automatically marked as stale because no requested feedback has been provided. It will be closed if no further activity occurs. Thank you for your contributions.