api(ticdc): add more verification for changefeed config (#12256)

[ti-chi-bot]

This is an automated cherry-pick of #12256

What problem does this PR solve?

Issue Number: close #12253

What is changed and how it works?

• Enhanced Changefeed Configuration Validation: Implemented more robust validation checks for changefeed configurations, covering both creation and update scenarios to prevent invalid setups.

This PR will validate the config of column selectors and dispatcher partition when creating a changefeed or resuming a changefeed. Avoid creating a failed changefeed if the downstream is Kafka.

Check List

Tests

• Unit test
• Integration test

Questions

Will it cause performance regression or break compatibility?

Do you need to update user documentation, design documentation or monitoring documentation?

Release note

Add more verification for the changefeed config when creating changefeed.

[gemini-code-assist[bot]]

Summary of Changes

Hello @ti-chi-bot, I'm Gemini Code Assist[^1]! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly improves the stability and reliability of TiCDC's changefeed management by introducing stricter validation for changefeed configurations. By proactively checking for invalid settings related to column selectors and Kafka dispatcher partitions during creation and updates, it prevents the deployment of misconfigured changefeeds that would otherwise fail, thereby enhancing the user experience and system robustness.

Highlights

• Enhanced Changefeed Configuration Validation: Implemented more robust validation checks for changefeed configurations during both creation and update operations. This prevents invalid setups, particularly for column selectors and Kafka dispatcher partitions, ensuring that changefeeds are not created or resumed with problematic configurations.
• Refactored Table Verification Logic: The internal getVerifiedTables helper function was introduced and integrated into the changefeed creation and update flows. This centralizes the logic for verifying tables against replication configurations, including sink scheme, topic, and protocol.
• Removal of ErrOldValueNotEnabled: The error ErrOldValueNotEnabled and its corresponding definition in errors.toml have been removed, streamlining error handling.
• Comprehensive Test Coverage: New unit and integration tests have been added and existing ones updated to cover the enhanced validation logic, specifically for invalid dispatcher configurations in Kafka sinks, ensuring the new checks function as expected.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with :thumbsup: and :thumbsdown: on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

[^1]: Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

[EmmaDuDu]

/retest

[ti-chi-bot[bot]]

@flowbehappy: adding LGTM is restricted to approvers and reviewers in OWNERS files.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[wk989898]

@benmeadowcroft PTAL.

[ti-chi-bot[bot]]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: flowbehappy, wk989898, yudongusa

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• ~~OWNERS~~ [wk989898]
• ~~cdc/api/v2/OWNERS~~ [yudongusa]

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[wk989898]

/retest

[wk989898]

/retest-required

[codecov[bot]]

Codecov Report

:x: Patch coverage is 77.77778% with 6 lines in your changes missing coverage. Please review. :warning: Please upload report for BASE (release-8.5@df4a904). Learn more about missing BASE report.

Additional details and impacted files

Components	Coverage Δ
cdc	57.7579% <0.0000%> (?)
dm	48.7795% <0.0000%> (?)
engine	50.6715% <0.0000%> (?)

Flag	Coverage Δ
unit	53.4326% <77.7777%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

@@               Coverage Diff                @@
##             release-8.5     #12463   +/-   ##
================================================
  Coverage               ?   53.4326%           
================================================
  Files                  ?       1003           
  Lines                  ?     136862           
  Branches               ?          0           
================================================
  Hits                   ?      73129           
  Misses                 ?      58273           
  Partials               ?       5460           

:rocket: New features to boost your workflow:

• :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[wk989898]

/retest-required