camel-kafka-connector icon indicating copy to clipboard operation
camel-kafka-connector copied to clipboard

Published 20 hours ago • verified publisher icon apache

Azure Camel Kafka Sink Connector - Avro File Format Issue

Open chethan-bs opened this issue 1 year ago • 3 comments

We are using Data Lake Sink connector for Azure storage account . We would like to store the files where it will be serialized in Avroformat. Is there any sample connector configurations that you can share for the same. @valdar

chethan-bs avatar Mar 19 '24 18:03 chethan-bs

Write your own SMT. It's the faster solution.

oscerd avatar Mar 19 '24 18:03 oscerd

@oscerd we have built custom smt , and post custom connect record , though we send avrobyte (Binary encoded) value , the connector sink worker is deserailizing when it creates the file for storage account. Is there any connector configuration which will send the custom record as such to Storage account . Is it something like marshal or unmarshal config or somethign else. Can you give an example connector config if in case you address this.

chethan-bs avatar Mar 22 '24 09:03 chethan-bs

The upload operation in the component will use an InputStream. Also without see what you're doing with the smt it's impossible to help.

oscerd avatar Mar 27 '24 10:03 oscerd