kafka-connect-elasticsearch icon indicating copy to clipboard operation
kafka-connect-elasticsearch copied to clipboard

Porting Scripted Upsert

Open NicholasDCole opened this issue 11 months ago • 2 comments

Add scripted upsert functionality to ES Sink Connector

Problem

Need scripted upsert functionality to support older Elastic Search client/server version

Also, need ability to inject Kafka payload as Params object

Solution

Add SCRIPTED_UPSERT write.method and custom logic in DataConverter.java to handle parameter injection as well as script injection into ES UpdateRequest

Does this solution apply anywhere else?
  • [ ] yes
  • [ ] no
If yes, where?

Test Strategy

Testing done:
  • [x] Unit tests
  • [ ] Integration tests
  • [ ] System tests
  • [x] Manual tests

Release Plan

Merge -> 11.1.x -> Create release

This is a Backport for 11.1.x . I will also merge to master within another PR

N/A

NicholasDCole avatar Mar 11 '24 20:03 NicholasDCole

CLA assistant check
All committers have signed the CLA.

cla-assistant[bot] avatar Mar 11 '24 20:03 cla-assistant[bot]

Please review @sp-gupta @kkonstantine @snehashisp

NicholasDCole avatar Mar 13 '24 13:03 NicholasDCole