kafka-connect-storage-common
kafka-connect-storage-common copied to clipboard
Allow configuration of NULL value replacement during Avro file creation
As of now, when the format class is set to format.class=io.confluent.connect.s3.format.avro.AvroFormat, the configuration value used for ignore.default.for.nullables is the default false. There seems to be no way to turn it to true since this configuration parameter is not referenced in the sink task configuration (as opposed to enhanced.avro.schema.support which can be turned ON or OFF).
This means that a null-value which has a default will have it set to the default in the Avro file dumped in S3 for example whereas the same record in Kafka will be properly set to NULL.
Could the sink task configuration be extended to support passing the ignore.default.for.nullables parameter down to the AvroFormat class (through AvroDataConfig apparently)?