tidb
tidb copied to clipboard
pingcap
*: collect subtask summary for `IMPORT INTO` tasks
What problem does this PR solve?
Issue Number: ref https://github.com/pingcap/tidb/issues/61088
Problem Summary:
What changed and how does it work?
What does this PR do:
- add collector for each step executor of
IMPORT INTOto collect summary. - store update time in summary. This value will be used to track the progress update time for each job.
Previously, imported row is stored in subtask meta, which is not easy to use. After this PR, we no longer need Result in subtask meta, row count can be calculated from all subtask summaries.
Check List
Tests
- [X] Unit test
- [ ] Integration test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test
- [ ] I checked and no code files have been changed.
This PR doesn't add new unit test, but extend existing tests to verfiy the functionality of each collector.
Side effects
- [ ] Performance regression: Consumes more CPU
- [ ] Performance regression: Consumes more Memory
- [ ] Breaking backward compatibility
Documentation
- [ ] Affects user behaviors
- [ ] Contains syntax changes
- [ ] Contains variable changes
- [ ] Contains experimental features
- [ ] Changes MySQL compatibility
Release note
Please refer to Release Notes Language Style Guide to write a quality release note.
None
Hi @joechenrh. Thanks for your PR.
PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.
I understand the commands that are listed here.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
![tiprow[bot] avatar](https://avatars.githubusercontent.com/in/18927?v=4 "Published by a pub.dev verified publisher"){kind=small}
Codecov Report
Attention: Patch coverage is 39.60396% with 122 lines in your changes missing coverage. Please review.
Project coverage is 75.6895%. Comparing base (
7487af5) to head (be955b1). Report is 25 commits behind head on master.
Additional details and impacted files
@@ Coverage Diff @@
## master #61600 +/- ##
================================================
+ Coverage 72.8941% 75.6895% +2.7954%
================================================
Files 1740 1794 +54
Lines 482921 499937 +17016
================================================
+ Hits 352021 378400 +26379
+ Misses 109294 98696 -10598
- Partials 21606 22841 +1235
| Flag | Coverage Δ | |
|---|---|---|
| integration | 50.1205% <0.0000%> (?) |
|
| unit | 73.0195% <40.0000%> (+0.8749%) |
:arrow_up: |
Flags with carried forward coverage won't be shown. Click here to find out more.
| Components | Coverage Δ | |
|---|---|---|
| dumpling | 52.7804% <ø> (ø) |
|
| parser | ∅ <ø> (∅) |
|
| br | 63.7431% <ø> (+17.4099%) |
:arrow_up: |
:rocket: New features to boost your workflow:
- :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
- :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
![codecov[bot] avatar](https://avatars.githubusercontent.com/in/254?v=4 "Published by a pub.dev verified publisher"){kind=small}
[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: D3Hunter, GMHDBJD
The full list of commands accepted by this bot can be found here.
The pull request process is described here
- ~~OWNERS~~ [D3Hunter,GMHDBJD]
- ~~pkg/ddl/OWNERS~~ [D3Hunter,GMHDBJD]
- ~~pkg/disttask/OWNERS~~ [D3Hunter,GMHDBJD]
- ~~pkg/executor/importer/OWNERS~~ [D3Hunter,GMHDBJD]
- ~~pkg/lightning/OWNERS~~ [D3Hunter,GMHDBJD]
Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment
![ti-chi-bot[bot] avatar](https://avatars.githubusercontent.com/in/214286?v=4 "Published by a pub.dev verified publisher"){kind=small}
[LGTM Timeline notifier]
Timeline:
2025-06-20 05:57:32.266489244 +0000 UTC m=+424104.989668224: :ballot_box_with_check: agreed by D3Hunter.2025-07-03 01:38:47.29445652 +0000 UTC m=+1531780.017635498: :ballot_box_with_check: agreed by GMHDBJD.
@joechenrh: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.
In response to this:
/retest
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
@joechenrh: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.
In response to this:
/retest
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
@joechenrh: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.
In response to this:
/retest
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
@joechenrh: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.
In response to this:
/retest
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
@joechenrh: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.
In response to this:
/retest
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
@joechenrh: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.
In response to this:
/retest
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
@joechenrh: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.
In response to this:
/retest
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
@joechenrh: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.
In response to this:
/retest
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
@joechenrh: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.
In response to this:
/retest
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
@joechenrh: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.
In response to this:
/retest
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
@joechenrh: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.
In response to this:
/retest
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
@joechenrh: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.
In response to this:
/retest
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
@joechenrh: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:
| Test name | Commit | Details | Required | Rerun command |
|---|---|---|---|---|
| non-block-pull-unit-test-next-gen | be955b1467ea57a95f9378635fb15b912337ec0a | link | false | /test pull-unit-test-next-gen |
Full PR test history. Your PR dashboard.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.