s3-connector-for-apache-kafka icon indicating copy to clipboard operation
s3-connector-for-apache-kafka copied to clipboard

Published 20 hours ago • verified publisher icon Aiven-Open

Unknown Magic byte

Open ram-venket36 opened this issue 5 months ago • 3 comments

Hi

I have below setup

Event Streams(Apache kafka platform) Apicurio schema registry Java spring custom app that produce avro serialized events using Apicurio SerDes library Aiven S3 sink connector configure to that topic.

With this implementation I am getting error as SerializationException: Unknown Magic byte when the connector tries to store the events in S3. With Java custom consumer app I am able to deserialize(Apicurio Serdes library) those events successfully but only via this connector getting issue.

Does this connector work with Apicurio schema registry or only with Confluent scheme registry ? Also events serialized by Apicurio SerDes library can be converted by Confluent AvroConverter ?

Value.converter.schema.registry.url : 'My apicurio schema registry url' value.converter: io.confluent.connect.avro.AvroConverter value.converter.use.latest.version: true value.converter.schema.enable: true value.converter.specific.avro.reader:true value.converter.enhanced.avro.schema.support: true topic : topic name Key.converter: ....Storage.StringConverter Connector.Class:...... AivenKafkaConnectS3SinkConnector

Apart from this I have given default S3 bucket, access details and broker URL.

Thanks

ram-venket36 avatar Aug 29 '24 13:08 ram-venket36