ticdc: Support `create table` ddl only appear in tidb_ddl_history instead of tidb_ddl_job table

[hongyunyan]

What problem does this PR solve?

Issue Number: close #10908

What is changed and how it works?

1. Support to listen to tidb_ddl_history table in ddlPuller
2. filter other ddls in tidb_ddl_history, and only choose create table ddl as the real ddl job
3. remove useless implementations, like listen to DDLJobList etc.

Check List

Tests

• Manual test (add detailed scripts or steps below)

1. test mannually with set global tidb_enable_fast_create_table =on and set global tidb_enable_fast_create_table =off. To make sure TiCDC can work well in both old ddl version and new ddl versions.
2. I compared the time it took for TiCDC to process 100 create table and add columns before and after the modification. Both took 5 mins. Thus, the modification will not affect TiCDC performance.

Questions

Will it cause performance regression or break compatibility?

Do you need to update user documentation, design documentation or monitoring documentation?

Release note

None

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 1.42857% with 69 lines in your changes are missing coverage. Please review.

Project coverage is 57.4968%. Comparing base (936b4d4) to head (7da4ec5). Report is 3 commits behind head on master.

Additional details and impacted files

Components	Coverage Δ
cdc	61.8129% <1.4285%> (+0.1429%)	:arrow_up:
dm	51.2679% <ø> (+0.0525%)	:arrow_up:
engine	63.4232% <ø> (+0.0423%)	:arrow_up:

Flag	Coverage Δ
unit	57.4968% <1.4285%> (+0.1072%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

@@               Coverage Diff                @@
##             master     #10907        +/-   ##
================================================
+ Coverage   57.3895%   57.4968%   +0.1072%     
================================================
  Files           851        851                
  Lines        125467     125840       +373     
================================================
+ Hits          72005      72354       +349     
- Misses        48044      48060        +16     
- Partials       5418       5426         +8     

[hongyunyan]

/test verify

[hongyunyan]

/test cdc-integration-mysql-test

[hongyunyan]

/test cdc-integration-pulsar-test

[hongyunyan]

/test dm-integration-test

[hongyunyan]

/test dm-compatibility-test

[hongyunyan]

/test cdc-integration-kafka-test

[hongyunyan]

/test cdc-integration-pulsar-test

[hongyunyan]

/test verify

[CharlesCheung96]

Please add some integration tests to cover the tidb_enable_fast_create_table=on case.

[hicqu]

Rest LGTM.

[hongyunyan]

/retest

[hongyunyan]

/retest

[hongyunyan]

/hold

[hongyunyan]

/test dm-integration-test

[ti-chi-bot[bot]]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: CharlesCheung96, hicqu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• ~~OWNERS~~ [CharlesCheung96,hicqu]

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot[bot]]

[LGTM Timeline notifier]

Timeline:

• 2024-04-17 06:59:31.246928433 +0000 UTC m=+407790.357974881: :ballot_box_with_check: agreed by hicqu.
• 2024-05-07 03:52:28.188678489 +0000 UTC m=+934101.945814060: :ballot_box_with_check: agreed by CharlesCheung96.

[hongyunyan]

/unhold

[hongyunyan]

/retest

[hongyunyan]

/test verify

[hongyunyan]

/test cdc-integration-mysql-test