kafka-connect-jdbc
kafka-connect-jdbc copied to clipboard
feat: adding pollingtimestamp metadata in bulk mode
Problem
While using bulk mode in JDBC source connector, polling time is not sent to the records metadata. Hence insert field smt does not work to add polling time in the records.
Also whenever the jdbc is restarted polling is started again results in reading the table again or duplicate data and make difficult to bifurcate data which is added in the queue at each poll.
Solution
adding timestamp helps in the bifurcation.
Does this solution apply anywhere else?
- [ ] yes
- [ ] no
If yes, where?
Test Strategy
end to end dataflow check
Testing done:
- [ ] Unit tests
- [ ] Integration tests
- [ ] System tests
- [ ] Manual tests
Release Plan
:tada: All Contributor License Agreements have been signed. Ready to merge.
:white_check_mark: sachin-kansal
Please push an empty commit if you would like to re-run the checks to verify CLA status for all contributors.
@confluent-antonio please review my Pull Request
@aniketshrimal , could please review my Pull Request
@confluent-antonio please review my Pull Request
I'm sorry, I'm not a developer. I can't review your PR.
Hi @sangeet259, @rhauch Could you please review my Pull request.
hi @petersilverwood, could you plase review my Pull Request.