kafka-connect-jdbc
kafka-connect-jdbc copied to clipboard
Required DB2DatabaseDialect to support `Insert or Ignore` Upsert style
Problem
We had a trigger that was processing on each inserted record and updates the record's status column to processed. While kafka-connect-jdbc retries with upsert it updates that status column back to pending;
DB2DatabaseDialect by default supports INSERT OR UPDATE but we required INSERT OR IGNORE
Fix for #874
Solution
- Add UpsertStyle configuration in JdbcSinkConfig
- Update Db2DatabaseDialect to read UpsertStyle config
- Update Db2DatabaseDialect to build the merge statement based on the UpsertStyle
Does this solution apply anywhere else?
- [x] yes
- [ ] no
If yes, where?
- This UpsertStyle configuration can be used by any other DatabaseDialect to provide support for
INSERT OR UPDATEandINSERT OR IGNOREUpsertStyle
Test Strategy
- If UpsertStyle is not provided this will be taken as default which means as it was working before this configuration. So there is no impact on an existing application using
kafka-connect-jdbc.
- Test config value if this configuration is not provided
- Test config value if this configuration is provided
- DB2DatabaseDialect to generate the merge statement based on this style.
- Test merge sql statement for all type of upsert combination when
upsert.style=insert_or_ignore
Testing done:
- [x] Unit tests
- [ ] Integration tests
- [ ] System tests
- [ ] Manual tests
Release Plan
We can merge it to master so that this functionality is available for everyone. Going forward all other DatabseDialect can also be updated to respect this configuration is possible.
It looks like @RajKumarMondol hasn't signed our Contributor License Agreement, yet.
The purpose of a CLA is to ensure that the guardian of a project's outputs has the necessary ownership or grants of rights over all contributions to allow them to distribute under the chosen licence. Wikipedia
You can read and sign our full Contributor License Agreement here.
Once you've signed reply with [clabot:check] to prove it.
Appreciation of efforts,
clabot
[clabot:check]
@confluentinc It looks like @RajKumarMondol just signed our Contributor License Agreement. :+1:
Always at your service,
clabot
Hi @rhauch, could you review this PR?
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
0 out of 2 committers have signed the CLA.
:x: Raj Kumar Mondol
:x: RajKumarMondol
Raj Kumar Mondol seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.
I had signed cla in past and again I have signed the latest versio of the CLA now. Cla link in email is not reachable should be updated.