Avro messages with Union type are not shown correctly

[christoph110]

Describe the bug Avro messages are not shown correctly when using Union type.

Set up Use provectuslabs/kafka-ui:0.4.0 image. Use confluentinc schema registry confluentinc/cp-schema-registry:7.0.1

version: '3.5'
...
services:
...
  schema-registry:
    image: confluentinc/cp-schema-registry:7.0.1
    environment:
      SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: zookeeper:2181
      SCHEMA_REGISTRY_HOST_NAME: schema-registry
      SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: kafka:9092
    ports:
      - "8081:8081"
    depends_on:
      - zookeeper
      - kafka
...
  kafka-ui:
    image: provectuslabs/kafka-ui:v0.4.0
    ports:
      - "8080:8080"
    depends_on:
      - schema-registry
    environment:
      KAFKA_CLUSTERS_0_NAME: local
      KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka:9092
      KAFKA_CLUSTERS_0_SCHEMAREGISTRY: http://schema-registry:8081
...

Steps to Reproduce Steps to reproduce the behavior:

1. Run docker compose
2. Send Avro message using confluent_kafka.schema_registry.avro.AvroSerializer

Expected behavior The message for an Avro schema that contains Union types should be displayed correctly.

Screenshots Using this schema:

Creates this rendered message:

Expected message rendering would be the same as for the consumer using the confluent_kafka.schema_registry.avro.AvroDeserializer module:

{'a': 'hello'}

Additional context

[github-actions[bot]]

Hello there christoph110! 👋

Thank you and congratulations 🎉 for opening your very first issue in this project! 💖

In case you want to claim this issue, please comment down below! We will try to get back to you as soon as we can. 👀

[Haarolean]

Hey, thanks for reaching out. We'll schedule the fix for current release.

[shubhwip]

@Haarolean Can i work on this ?

[Haarolean]

@shubhwip please keep in mind that we're gonna change serdes a lot within #2125, so it might be a good idea to postpone it and do a PR for the new codebase

[shubhwip]

@shubhwip please keep in mind that we're gonna change serdes a lot within #2125, so it might be a good idea to postpone it and do a PR for the new codebase

Okay sure will wait for #2125 to get merged.

[iliax]

Hello @christoph110,

This behaviour is actually expected. Current json rendering policy is following json encoding spec for avro (https://avro.apache.org/docs/1.11.1/specification/_print/#json-encoding).

Reason why this is required is that for some types like int-long, string-bytes, float-double it is not possible to infer what exact type should be set in avro from json. I mean, for some simple cases like yours it can be done, but this is not universal.

I agree that we can make it configurable (or do it only when it is possible), but we should think about it more carefully.

PS. we should also check protobuf oneof fields, since they have pretty similar sttucture.

cc @Haarolean @shubhwip

[christoph110]

Hi @iliax ,

that makes sense, didn't know about these specs. Thanks for looking into this. In this case, since it is an Avro specification and not related to kafka-ui, I would say this issue can be closed unless you want to keep it for further investigations (e.g. what you wrote under PS: )