flink-connector-elasticsearch
flink-connector-elasticsearch copied to clipboard
[FLINK-36143][ES6][ES7] Intro retry-on-conflict param to resolve Sink ES occurred "version conflict"
When I used ES connector Write update datas, it occurred "version conflict", like this " org.elasticsearch.index.engine.VersionConflictEngineException: [project][1140860]: version conflict, current version [5] is different than the one provided [4] " When we set the conf failure-handler = ignore, It lead to lost data, and if set the conf failure-handler = fail, job will restore. this cost is too heavy. maybe second try it will success.
so I intro this conf param, this will make job failed cost smaller.
Thanks for opening this pull request! Please check out our contributing guidelines. (https://flink.apache.org/contributing/how-to-contribute.html)
@schulzp @alpinegizmo @aljoscha @rmetzger Has any body to review my code? thk ❤️❤️
Thanks for your advices,and I fixed this. Has anybody can review my code?
Thank you for your patch! I am not an official maintainer. Your changes look reasonable, however, they lack test coverage. One other thing I noticed: The newly introduced constant
RETRY_ON_CONFLICT_NUMdoes not follow the naming convention of other config options inElasticsearchConnectorOptions: Most of them appear to end on[...]_OPTIONbut do not have the type-indicating suffix. Renaming it tosink.retries-on-conflictcould eliminate the confusion wether this is a boolean flag or a number.
@reswqa @mtfelisb hi guys,could you review my pr? thx
@rmetzger hi, thx for your triggering. I see there has some error when run IT case. And there also has same error in before PR . Is there some problems in project? The leatest merged pr doesn't have this step, latest un-merged PRs both have this error.