tidb
tidb copied to clipboard
pingcap
copr: increase extra concurrency for small coprocessor tasks
What problem does this PR solve?
Issue Number: close #37724
Problem Summary:
What is changed and how it works?
When there are many small coprocessor tasks, the default value of tidb_distsql_scan_concurrency harms the latency. This PR increases extra concurrency when we exactly know there are small coprocessor tasks.
Check List
Tests
- [x] Unit test
- [ ] Integration test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No code
Side effects
- [ ] Performance regression: Consumes more CPU
- [ ] Performance regression: Consumes more Memory
- [ ] Breaking backward compatibility
Documentation
- [ ] Affects user behaviors
- [ ] Contains syntax changes
- [ ] Contains variable changes
- [ ] Contains experimental features
- [ ] Changes MySQL compatibility
Release note
Please refer to Release Notes Language Style Guide to write a quality release note.
Optimize handles read requests by increasing extra concurrency for small coprocessor tasks.
[REVIEW NOTIFICATION]
This pull request has been approved by:
- guo-shaoge
- wshwsh12
To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.
The full list of commands accepted by this bot can be found here.
Reviewer can indicate their review by submitting an approval review. Reviewer can cancel approval by submitting a request changes review.
This pull request has been accepted and is ready to merge.
TiDB MergeCI notify
🔴 Bad News! New failing [1] after this pr merged. These new failed integration tests seem to be caused by the current PR, please try to fix these new failed integration tests, thanks!
| CI Name | Result | Duration | Compare with Parent commit |
|---|---|---|---|
| idc-jenkins-ci/integration-cdc-test | 🟥 failed 1, success 36, total 37 | 26 min | New failing |
| idc-jenkins-ci-tidb/integration-common-test | 🔴 failed 3, success 14, total 17 | 13 min | Existing failure |
| idc-jenkins-ci-tidb/tics-test | 🔴 failed 1, success 0, total 1 | 4 min 34 sec | Existing failure |
| idc-jenkins-ci-tidb/integration-ddl-test | 🟢 all 6 tests passed | 23 min | Existing passed |
| idc-jenkins-ci-tidb/common-test | 🟢 all 11 tests passed | 10 min | Existing passed |
| idc-jenkins-ci-tidb/sqllogic-test-1 | 🟢 all 26 tests passed | 8 min 26 sec | Existing passed |
| idc-jenkins-ci-tidb/sqllogic-test-2 | 🟢 all 28 tests passed | 4 min 16 sec | Existing passed |
| idc-jenkins-ci-tidb/mybatis-test | 🟢 all 1 tests passed | 3 min 3 sec | Existing passed |
| idc-jenkins-ci-tidb/integration-compatibility-test | 🟢 all 1 tests passed | 2 min 54 sec | Existing passed |
| idc-jenkins-ci-tidb/plugin-test | 🟢 build success, plugin test success | 4min | Existing passed |
Any bench result to show the great improvement of this change?
I attached the bench result to the description.
Any bench result to show the great improvement of this change?
I attached the bench result to the description.
When there many requests has fan out characteristics, the test result including p99 should be much more better?
Any bench result to show the great improvement of this change?
I attached the bench result to the description.
When there many requests has fan out characteristics, the test result including p99 should be much more better?
@zhangjinpeng1987 The average latency and p99 should both be better if the coprocessor tasks are not accumulated on the tikv side, and this PR just optimizes the task wait on the tidb side. If there's already task queueing on the tikv side, this change may increase latency.