pingcap
best-practices: add a new guide on mastering partitioned tables
- Query optimization with partition pruning
- Performance comparison: Non-Partitioned vs Local Index vs Global Index
- Data cleanup efficiency: TTL vs DROP PARTITION
- Partition drop performance: Local Index vs Global Index
- Strategies to mitigate write hotspot issues with hash/key partitioning
- Partition management challenges and best practices
- Avoiding read/write hotspots on new partitions
- Using PRE_SPLIT_REGIONS, SHARD_ROW_ID_BITS, and region splitting
- Converting between partitioned and non-partitioned tables
- Batch DML, Pipeline DML, IMPORT INTO, and Online DDL efficiency comparison
First-time contributors' checklist
- [ ] I've signed the Contributor License Agreement, which is required for the repository owners to accept my contribution.
What is changed, added or deleted? (Required)
Which TiDB version(s) do your changes apply to? (Required)
Tips for choosing the affected version(s):
By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.
For details, see tips for choosing the affected versions.
- [x] master (the latest development version)
- [ ] v9.0 (TiDB 9.0 versions)
- [x] v8.5 (TiDB 8.5 versions)
- [ ] v8.1 (TiDB 8.1 versions)
- [ ] v7.5 (TiDB 7.5 versions)
- [ ] v7.1 (TiDB 7.1 versions)
- [ ] v6.5 (TiDB 6.5 versions)
- [ ] v6.1 (TiDB 6.1 versions)
- [ ] v5.4 (TiDB 5.4 versions)
What is the related PR or file link(s)?
- This PR is translated from:
- Other reference link(s):
Do your changes match any of the following descriptions?
- [ ] Delete files
- [ ] Change aliases
- [ ] Need modification after applied to another branch
- [ ] Might cause conflicts after applied to another branch
Summary of Changes
Hello @shaoxiqian, I'm Gemini Code Assist[^1]! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request introduces a comprehensive guide on mastering partitioned tables in TiDB. It covers critical aspects such as optimizing query performance through partition pruning and global indexes, efficiently managing bulk data deletion with DROP PARTITION versus TTL, mitigating write hotspots using hash/key partitioning, and best practices for managing new range partitions. The guide also provides a detailed comparison of methods for converting tables between partitioned and non-partitioned states, offering practical insights and recommendations for effective TiDB environment management.
Highlights
- Query Optimization: Details how partition pruning enhances query efficiency and compares performance across non-partitioned, local index, and global index configurations, providing insights into when to use each.
- Bulk Data Deletion: Explores the efficiency of data cleanup using TTL versus direct partition drops, highlighting the significant performance advantages of
DROP PARTITIONand its implications for local vs. global indexes. - Write Hotspot Mitigation: Discusses strategies, particularly hash/key partitioning, to alleviate write hotspot issues caused by monotonically increasing primary keys, improving load distribution and throughput.
- Partition Management: Addresses common challenges like read and write hotspots in new range partitions and provides solutions using
SHARD_ROW_ID_BITS,PRE_SPLIT_REGIONS, and region splitting for bothNONCLUSTEREDandCLUSTEREDtables. - Table Conversion Methods: Compares the efficiency of various methods (Batch DML, Pipeline DML, IMPORT INTO, Online DDL) for converting between partitioned and non-partitioned tables, offering recommendations for optimal performance.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in pull request comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with :thumbsup: and :thumbsdown: on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
[^1]: Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.
![gemini-code-assist[bot] avatar](https://avatars.githubusercontent.com/in/956858?v=4 "Published by a pub.dev verified publisher"){kind=small}
[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: Once this PR has been reviewed and has the lgtm label, please ask for approval from hfxsd. For more information see the Code Review Process. Please ensure that each of them provides their approval before proceeding.
The full list of commands accepted by this bot can be found here.
Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment
![ti-chi-bot[bot] avatar](https://avatars.githubusercontent.com/in/214286?v=4 "Published by a pub.dev verified publisher"){kind=small}
[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: Once this PR has been reviewed and has the lgtm label, please ask for approval from hfxsd. For more information see the Code Review Process. Please ensure that each of them provides their approval before proceeding.
The full list of commands accepted by this bot can be found here.
Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment
From the checks:
Error: R] [file:///home/runner/work/docs/docs/auto_increment.md](file:///home/runner/work/docs/docs/auto_increment.md) | Cannot find file
I think the file is called auto-increment.md instead of auto_increment.md.
tidb_partitioned_tables_guide.md: 18: MD007/ul-indent Unordered list indentation [Expected: 4; Actual: 2]
tidb_partitioned_tables_guide.md: 21: MD007/ul-indent Unordered list indentation [Expected: 4; Actual: 2]
tidb_partitioned_tables_guide.md: 25: MD007/ul-indent Unordered list indentation [Expected: 4; Actual: 2]
tidb_partitioned_tables_guide.md: 55: MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "- Non-Partitioned Table"]
tidb_partitioned_tables_guide.md: 110: MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "- The **Average Query Time** w..."]
tidb_partitioned_tables_guide.md: 114: MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "- **Average Query Time**: from..."]
tidb_partitioned_tables_guide.md: 138: MD031/blanks-around-fences Fenced code blocks should be surrounded by blank lines [Context: "```yaml"]
tidb_partitioned_tables_guide.md: 147: MD031/blanks-around-fences Fenced code blocks should be surrounded by blank lines [Context: "```yaml"]
tidb_partitioned_tables_guide.md: 153: MD031/blanks-around-fences Fenced code blocks should be surrounded by blank lines [Context: "```"]
tidb_partitioned_tables_guide.md: 221: MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "- On a write-heavy table, TTL ..."]
tidb_partitioned_tables_guide.md: 227: MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "- DROP PARTITION removes an en..."]
tidb_partitioned_tables_guide.md: 232: MD034/no-bare-urls Bare URL used [Context: "https://docs.pingcap.com/tidb/..."]
tidb_partitioned_tables_guide.md: 423: MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "- When a new partition is crea..."]
tidb_partitioned_tables_guide.md: 427: MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "- Queries using **Point Get** ..."]
tidb_partitioned_tables_guide.md: 430: MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "- Suitable for workloads where..."]
tidb_partitioned_tables_guide.md: 516: MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "- Queries using **Point Get** ..."]
tidb_partitioned_tables_guide.md: 519: MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "- **Manual region splitting** ..."]
tidb_partitioned_tables_guide.md: 522: MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "- Ideal when low-latency point..."]
tidb_partitioned_tables_guide.md: 601: MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "- **No hotspot risk from new p..."]
tidb_partitioned_tables_guide.md: 605: MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "- **Cannot use DROP PARTITION*..."]
tidb_partitioned_tables_guide.md: 608: MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "- Best suited for use cases th..."]
tidb_partitioned_tables_guide.md: 629: MD026/no-trailing-punctuation Trailing punctuation in heading [Punctuation: '.']
tidb_partitioned_tables_guide.md: 631: MD022/blanks-around-headings/blanks-around-headers Headings should be surrounded by blank lines [Expected: 1; Actual: 0; Below] [Context: "#### Table Schema: `fa`"]
tidb_partitioned_tables_guide.md: 632: MD031/blanks-around-fences Fenced code blocks should be surrounded by blank lines [Context: "```sql"]
tidb_partitioned_tables_guide.md: 653: MD012/no-multiple-blanks Multiple consecutive blank lines [Expected: 1; Actual: 2]
tidb_partitioned_tables_guide.md: 654: MD022/blanks-around-headings/blanks-around-headers Headings should be surrounded by blank lines [Expected: 1; Actual: 0; Below] [Context: "#### Table Schema: `fa_new`"]
tidb_partitioned_tables_guide.md: 655: MD031/blanks-around-fences Fenced code blocks should be surrounded by blank lines [Context: "```sql"]
tidb_partitioned_tables_guide.md: 670: MD022/blanks-around-headings/blanks-around-headers Headings should be surrounded by blank lines [Expected: 1; Actual: 0; Below] [Context: "#### Description"]
tidb_partitioned_tables_guide.md: 672: MD022/blanks-around-headings/blanks-around-headers Headings should be surrounded by blank lines [Expected: 1; Actual: 0; Above] [Context: "### Method 1: By Default"]
tidb_partitioned_tables_guide.md: 680: MD012/no-multiple-blanks Multiple consecutive blank lines [Expected: 1; Actual: 2]
tidb_partitioned_tables_guide.md: 681: MD026/no-trailing-punctuation Trailing punctuation in heading [Punctuation: '.']
tidb_partitioned_tables_guide.md: 691: MD026/no-trailing-punctuation Trailing punctuation in heading [Punctuation: '.']
These mostly need some extra lines around lists etc.
[LGTM Timeline notifier]
Timeline:
2025-10-03 08:14:25.172266392 +0000 UTC m=+418045.428997792: :heavy_multiplication_x::repeat: reset by dveeden.2025-10-14 11:57:15.919784395 +0000 UTC m=+181741.997036955: :heavy_multiplication_x::repeat: reset by dveeden.2025-10-15 07:23:04.318583067 +0000 UTC m=+251690.395835627: :ballot_box_with_check: agreed by dveeden.
@gemini-code-assist review this PR as a technical writer. Revamp the structure of sections if needed, to make the article more readable.
@shaoxiqian: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:
| Test name | Commit | Details | Required | Rerun command |
|---|---|---|---|---|
| pull-verify | 8c5fc06a1f140a6d05c8c90675de16caa2c398a9 | link | true | /test pull-verify |
Full PR test history. Your PR dashboard.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.