Kafka retry mdclp

[djordje-mijatovic]

Checklist

• [ ] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
• [ ] Links to related issues (if applicable)
• [ ] Tests for the changes have been added/updated (if applicable)
• [ ] Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
• [ ] For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

Split Kafka Listener from MetadataChangeLogProcessor into 4 different Listeners (as there are 4 hooks now) and add separated Retry mechanism to each of them.

[github-actions[bot]]

Unit Test Results (build & test)

633 tests  ±0   629 :heavy_check_mark: ±0   15m 57s :stopwatch: +10s 162 suites ±0       4 :zzz: ±0  162 files   ±0       0 :x: ±0 

Results for commit 43bbc599. ± Comparison against base commit eeb7a9df.

:recycle: This comment has been updated with latest results.

[jjoyce0510]

(This is under review by Acryl team)

[pedro93]

Hello @djordje-mijatovic

Thank you for your PR. Could you please add change the PR title to match our guidelines and also add a description what you are trying to accomplish so we have a good git log in case future contributors need to understand?

Please see: https://github.com/datahub-project/datahub/blob/master/docs/CONTRIBUTING.md#commit-message-format for our guidelines.

[jjoyce0510]

Apologies for the delay @djordje-mijatovic!

We are taking a look at the latest shortly.

[RyanHolstien]

Considering the significant architectural changes this PR proposes, we need time to evaluate the impact on deployments and needs to be done carefully with a migration plan. Other pressing work currently takes priority over this, but we have it on our priority list.

[jjoyce0510]

Going to be closing this one for now. Let's reopen if it becomes relevant again.

Cheers John