kafka-connect-elastic-sink icon indicating copy to clipboard operation
kafka-connect-elastic-sink copied to clipboard

Push message to specific elastic index based on values of one or more fields of the record

Open rohankapoor1 opened this issue 5 years ago • 0 comments

I was thinking to propose a new configuration (like record.index.map) while setting up the connector where user can give a record field or combination of record fields which can be used to push this record to a particular ES index.

Eg: Record = {"fieldA" : "valueA", "fieldB" : "valueB", "fieldC" : "valueC", "fieldD" : "valueD"}

record.index.map = [fieldA]-[fieldC]

So, this record will be stored into ES index named: valueA-valueC As it will be a default configuration provided so any external Kafka connect plugin like RegexRouter and TimestampRouter may not be needed. And I am thinking this will be useful for many consumers. What's your thought on this @skynyrd ? Also, can you please add a license file to the project. It would be nice.

rohankapoor1 avatar Jul 25 '19 17:07 rohankapoor1