pinot icon indicating copy to clipboard operation
pinot copied to clipboard

ConfluentSchemaRegistry IntegrationTest prints huge logs

Open walterddr opened this issue 3 years ago • 1 comments
trafficstars

https://github.com/apache/pinot/runs/7890686433?check_suite_focus=true

2022-08-18T02:12:33.6547446Z [INFO] Running org.apache.pinot.integration.tests.KafkaConfluentSchemaRegistryAvroMessageDecoderRealtimeClusterIntegrationTest
...
2022-08-18T02:14:26.6174836Z 02:14:25.310 ERROR [KafkaConfluentSchemaRegistryAvroMessageDecoder] [mytable__0__0__20220818T0214Z] Caught exception while decoding row in topic KafkaConfluentSchemaRegistryAvroMessageDecoderRealtimeClusterIntegrationTest, discarding row
2022-08-18T02:14:26.6175975Z org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id -1
2022-08-18T02:14:26.6176584Z Caused by: org.apache.kafka.common.errors.SerializationException: Unknown magic byte!
2022-08-18T02:14:26.6177510Z 02:14:25.867 ERROR [KafkaConfluentSchemaRegistryAvroMessageDecoder] [mytable__1__0__20220818T0214Z] Caught exception while decoding row in topic KafkaConfluentSchemaRegistryAvroMessageDecoderRealtimeClusterIntegrationTest, discarding row
2022-08-18T02:14:26.6178539Z org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id -1
2022-08-18T02:14:26.6179139Z Caused by: org.apache.kafka.common.errors.SerializationException: Unknown magic byte!
...

going on for 120,000 lines.

walterddr avatar Aug 18 '22 13:08 walterddr

CC @KKcorps ^ since we recently also added the schema registry in protobuf test

walterddr avatar Aug 18 '22 13:08 walterddr

2022-08-18T02:14:26.6175975Z org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id -1
2022-08-18T02:14:26.6176584Z Caused by: org.apache.kafka.common.errors.SerializationException: Unknown magic byte!

This happens when trying to use the confluent schema registry to deserialize a message that was written using the standard (non-schemaregistry) avro serializer. I'll take a quick look at this.

agavra avatar Oct 03 '22 15:10 agavra