pulsar icon indicating copy to clipboard operation
pulsar copied to clipboard

[fix][io] Upgrade Debezium connector version to 2.6.1.Final

Open mukesh154 opened this issue 1 year ago • 4 comments

Fixes #22626

Motivation

To Upgrade debezium version to 2.6.1.Final for all source connectors.

It will address this comment as well.

Modifications

Upgraded debezium version from 1.9.7.Final to 2.6.1.Final all debezium source connectors which includes mysql, postgresql, mongodb, mssql & oracle.

Verifying this change

  • [x] Make sure that the change passes the CI checks.

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • [x] Dependencies (add or upgrade a dependency)
  • [ ] The public API
  • [ ] The schema
  • [ ] The default values of configurations
  • [ ] The threading model
  • [ ] The binary protocol
  • [ ] The REST endpoints
  • [ ] The admin CLI options
  • [ ] The metrics
  • [ ] Anything that affects deployment

Documentation

Check the box below or label this PR directly (if you have committer privilege).

Need to update docs?

  • [ ] doc
  • [ ] doc-required
  • [x] doc-not-needed
  • [ ] doc-complete

mukesh154 avatar Jul 26 '24 06:07 mukesh154

I'd recommend running tests for this PR in your own fork so that you get CI feedback without consuming apache/pulsar CI resources.

apache/pulsar pull requests should be first tested in your own fork since the apache/pulsar CI based on GitHub Actions has constrained resources and quota. GitHub Actions provides separate quota for pull requests that are executed in a forked repository.

  1. Go to https://github.com/cognitree/pulsar/tree/master and ensure that your master branch is up to date with https://github.com/apache/pulsar
    Sync your fork if it's behind.
  2. Open a pull request to your own fork. You can use this link to create the pull request in your own fork: Create PR in fork for running tests
  3. Edit the description of the pull request https://github.com/apache/pulsar/pull/23078 and add the link to the PR that you opened to your own fork so that the reviewer can verify that tests pass in your own fork.
  4. Ensure that tests pass in your own fork. Your own fork will be used to run the tests during the PR review and any changes made during the review. You as a PR author are responsible for following up on test failures. Please report any flaky tests as new issues at https://github.com/apache/pulsar/issues after checking that the flaky test isn't already reported.
  5. When the PR is approved, it will be possible to restart the Pulsar CI workflow within apache/pulsar repository by adding a comment "/pulsarbot rerun-failure-checks" to the PR. An alternative for the PR approval is to add a ready-to-test label to the PR. This can be done by Apache Pulsar committers.
  6. When tests pass on the apache/pulsar side, the PR can be merged by a Apache Pulsar Committer.

lhotari avatar Jul 30 '24 07:07 lhotari

The PR checks passed on my fork for 2.6.1.Final. Check here.

mukesh154 avatar Jul 30 '24 14:07 mukesh154

Since upgrading Debezium will be a breaking change. We need to document the upgrade notes for our users. What is the upgrade path and instructions for existing installations? Is it possible to upgrade an existing installation without loosing events etc.?

lhotari avatar Jul 30 '24 16:07 lhotari

Here is the doc for the breaking changes.

mukesh154 avatar Aug 20 '24 16:08 mukesh154

Superseded by #24712

lhotari avatar Sep 07 '25 08:09 lhotari