tiflow icon indicating copy to clipboard operation
tiflow copied to clipboard

tracker(dm): schema tracker without embedded TiDB

Open lance6716 opened this issue 3 years ago • 10 comments

What problem does this PR solve?

Issue Number: close https://github.com/pingcap/tiflow/issues/5334

What is changed and how it works?

Exec now accepts ast.StmtNode, and SchemaTracker should do the preprocessing works like TiDB executor.

I hope other API is not effected

Check List

Tests

  • pass old tests

Questions

Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?

Release note

DM no longer uses embedded TiDB to track table structure from upstream DDL

lance6716 avatar Aug 08 '22 09:08 lance6716

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • D3Hunter
  • GMHDBJD

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment. After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review. Reviewer can cancel approval by submitting a request changes review.

ti-chi-bot avatar Aug 08 '22 09:08 ti-chi-bot

/run-dm-integration-test

lance6716 avatar Aug 09 '22 08:08 lance6716

/run-dm-integration-test

lance6716 avatar Aug 09 '22 09:08 lance6716

now new SchemaTracker will not report error like "unsupported add column 'c' constraint UNIQUE KEY", many tests need to be updated :joy:

/run-dm-integration-test

lance6716 avatar Aug 09 '22 11:08 lance6716

@lance6716: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

ti-chi-bot avatar Aug 10 '22 12:08 ti-chi-bot

/run-dm-integration-test

lance6716 avatar Aug 11 '22 07:08 lance6716

/run-dm-integration-test

lance6716 avatar Aug 11 '22 08:08 lance6716

/run-dm-integration-test

lance6716 avatar Aug 11 '22 09:08 lance6716

/run-dm-integration-test

lance6716 avatar Aug 11 '22 11:08 lance6716

https://ci2.pingcap.net/blue/organizations/jenkins/dm_ghpr_integration_test/detail/dm_ghpr_integration_test/8816/pipeline

DM integration test passed. Most changes are test files, ptal @GMHDBJD @D3Hunter @lichunzhu

lance6716 avatar Aug 11 '22 13:08 lance6716

Codecov Report

Merging #6659 (6ba9d60) into master (2ea67ba) will increase coverage by 0.0423%. The diff coverage is 65.2107%.

Flag Coverage Δ
cdc 65.8342% <61.4781%> (-0.0339%) :arrow_down:
dm 52.0204% <62.5570%> (-0.0214%) :arrow_down:
engine 62.7071% <85.4922%> (+0.1709%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

@@               Coverage Diff                @@
##             master      #6659        +/-   ##
================================================
+ Coverage   59.2138%   59.2561%   +0.0423%     
================================================
  Files           780        785         +5     
  Lines         88405      89162       +757     
================================================
+ Hits          52348      52834       +486     
- Misses        31415      31662       +247     
- Partials       4642       4666        +24     

codecov-commenter avatar Aug 15 '22 02:08 codecov-commenter

/cc @GMHDBJD @lichunzhu

lance6716 avatar Aug 15 '22 07:08 lance6716

PTAL @GMHDBJD @lichunzhu

lance6716 avatar Aug 16 '22 02:08 lance6716

/merge

lance6716 avatar Aug 16 '22 03:08 lance6716

This pull request has been accepted and is ready to merge.

Commit hash: 4a6381b58562b0a366c9ac455ba5db683948d16a

ti-chi-bot avatar Aug 16 '22 03:08 ti-chi-bot

/run-dm-integration-test /run-integration-test /run-dm-compatibility-test

lance6716 avatar Aug 16 '22 06:08 lance6716

/run-integration-test /run-dm-compatibility-test

lance6716 avatar Aug 16 '22 06:08 lance6716

/run-dm-integration-test /run-integration-test

lance6716 avatar Aug 16 '22 07:08 lance6716

@lance6716: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot avatar Aug 16 '22 07:08 ti-chi-bot

/run-all-tests

lance6716 avatar Aug 16 '22 08:08 lance6716

/run-verify

lance6716 avatar Aug 16 '22 09:08 lance6716