hyperdrive icon indicating copy to clipboard operation
hyperdrive copied to clipboard

Rename prefix for additional properties for KafkaStreamReader

Open kevinwallimann opened this issue 5 years ago • 1 comments

Currently, additional properties for the KafkaStreamReader have to be specified with the prefix reader.options. E.g. reader.options.kafka.security.protocol or reader.options.kafka.ssl.key.password

This prefix is inconsistent with all other properties which start with decoder.avro., writer.parquet., manager.checkpoint or transformer.columns.. The properties for the KafkaStreamReader should start with reader.kafka., i.e. reader.kafka.options.kafka.security.protocol.

For additional properties like reader.options.failOnDataLoss or reader.options.minPartitions it's hard to find out which reader implementation they belong to.

Tasks

  • Change prefix for additional properties for KafkaStreamReader to reader.kafka.options.

How to migrate

  • All property keys starting with reader.options have to be replaced by reader.kafka.options

kevinwallimann avatar Feb 10 '20 10:02 kevinwallimann

Removed milestone v3.x.0 to reduce number of breaking changes for v3.0.0. This change is not urgent as long as there is no conflict with other readers.

kevinwallimann avatar Feb 19 '20 13:02 kevinwallimann