tidb [DNM]GC: Support keyspace level GC

What problem does this PR solve?

Issue Number: ref https://github.com/pingcap/tidb/issues/51593

Problem Summary:

What changed and how does it work?

Check List

Tests

[x] Manual test (add detailed scripts or steps below)

set keyspace level gc when create Keyspace: 1.1. create new keyspace with keyspace level gc:

curl -H "Content-Type: application/json" -X POST -d \
'{
    "name": "ks1",
    "config": {
        "gc_management_type": "keyspace_level_gc"
    }
}' "http://127.0.0.1:2379/pd/api/v2/keyspaces"

Get keyspace meta:

curl "http://127.0.0.1:2379/pd/api/v2/keyspaces/ks1"
{
    "id": 1,
    "name": "ks1",
    "state": "ENABLED",
    "created_at": 1712126594,
    "state_changed_at": 1712126594,
    "config": {
        "gc_management_type": "keyspace_level_gc"
    }
}

1.2. Start TiDB and check the etcd key You can see that the ETCD key of safe point and min start ts in pd is separated by Keyspace:

1.minstartts
/keyspaces/tidb/1/tidb/server/minstartts/1fbc785c-dada-4152-a2e7-f0534a386e6e

2.saved safe point
/keyspaces/tidb/1/tidb/store/gcworker/saved_safe_point

4.gc safe point 
/pd/7340112560849754553/keyspaces/gc_safe_point/00000001

5.service safe point 
/pd/7340112560849754553/keyspaces/service_safe_point/00000001/gc_worker

Check that PD's ETCD key remains in its original format without Keyspace set:

1. minstartts
/tidb/server/minstartts/f8545873-a97a-4aaf-be10-53633490b339

2. saved safe point 
/tidb/store/gcworker/saved_safe_point

3. gc safe point
/pd/7340566695597925295/gc/safe_point

4. service safe point 
/pd/7340566695597925295/gc/safe_point/service/gc_worker

Documentation

[ ] Affects user behaviors
[ ] Contains syntax changes
[ ] Contains variable changes
[ ] Contains experimental features
[ ] Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

Support TiDB server with different keyspace to advance GC safe point separately

Feb 26 '24 02:02 ystaticy

Hi @ystaticy. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Feb 26 '24 02:02 tiprow[bot]

Codecov Report

Attention: Patch coverage is 48.02867% with 145 lines in your changes missing coverage. Please review.

Project coverage is 74.2527%. Comparing base (0f0418f) to head (c543d86). Report is 916 commits behind head on master.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #51300        +/-   ##
================================================
+ Coverage   72.4015%   74.2527%   +1.8512%     
================================================
  Files          1483       1524        +41     
  Lines        429016     442233     +13217     
================================================
+ Hits         310614     328370     +17756     
+ Misses        99108      93204      -5904     
- Partials      19294      20659      +1365

Flag	Coverage Δ
integration	`50.1939% <31.8996%> (?)`
unit	`71.2173% <42.6523%> (-0.0304%)`	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	`53.9957% <ø> (ø)`
parser	`∅ <ø> (∅)`
br	`52.7771% <ø> (+11.5655%)`	:arrow_up:

Mar 06 '24 17:03 codecov[bot]

/test check-dev2

Mar 07 '24 09:03 ystaticy

@ystaticy: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test check-dev2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Mar 07 '24 09:03 tiprow[bot]

/ok-to-test

Mar 07 '24 10:03 tiancaiamao

/test unit-test

Apr 22 '24 13:04 ystaticy

@ystaticy: The specified target(s) for /test were not found. The following commands are available to trigger required jobs:

/test fast_test_tiprow
/test tidb_parser_test

Use /test all to run all jobs.

In response to this:

/test unit-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Apr 22 '24 13:04 tiprow[bot]

/test unit-test

Apr 22 '24 14:04 ystaticy

@ystaticy: The specified target(s) for /test were not found. The following commands are available to trigger required jobs:

/test fast_test_tiprow
/test tidb_parser_test

Use /test all to run all jobs.

In response to this:

/test unit-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Apr 22 '24 14:04 tiprow[bot]

/retest

Apr 23 '24 10:04 hawkingrei

/test check-dev2

Apr 26 '24 07:04 ystaticy

@ystaticy: The specified target(s) for /test were not found. The following commands are available to trigger required jobs:

/test fast_test_tiprow
/test tidb_parser_test

Use /test all to run all jobs.

In response to this:

/test check-dev2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Apr 26 '24 07:04 tiprow[bot]

/test all

Apr 26 '24 09:04 ystaticy

/test fast_test_tiprow

Apr 29 '24 10:04 ystaticy

@ystaticy: The specified target(s) for /test were not found. The following commands are available to trigger required jobs:

/test build
/test check-dev
/test check-dev2
/test mysql-test
/test pull-br-integration-test
/test pull-integration-ddl-test
/test pull-lightning-integration-test
/test pull-mysql-client-test
/test unit-test

The following commands are available to trigger optional jobs:

/test canary-notify-when-compatibility-sections-changed
/test pingcap/tidb/canary_ghpr_unit_test
/test pull-common-test
/test pull-e2e-test
/test pull-integration-common-test
/test pull-integration-copr-test
/test pull-integration-jdbc-test
/test pull-integration-mysql-test
/test pull-integration-nodejs-test
/test pull-sqllogic-test
/test pull-tiflash-test

Use /test all to run the following jobs that were automatically triggered:

pingcap/tidb/ghpr_build
pingcap/tidb/ghpr_check
pingcap/tidb/ghpr_check2
pingcap/tidb/ghpr_mysql_test
pingcap/tidb/ghpr_unit_test
pingcap/tidb/pull_integration_ddl_test
pingcap/tidb/pull_mysql_client_test

In response to this:

/test fast_test_tiprow

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Apr 29 '24 10:04 ti-chi-bot[bot]

/test tidb_parser_test

Apr 29 '24 10:04 ystaticy

@ystaticy: The specified target(s) for /test were not found. The following commands are available to trigger required jobs:

/test build
/test check-dev
/test check-dev2
/test mysql-test
/test pull-br-integration-test
/test pull-integration-ddl-test
/test pull-lightning-integration-test
/test pull-mysql-client-test
/test unit-test

The following commands are available to trigger optional jobs:

/test canary-notify-when-compatibility-sections-changed
/test pingcap/tidb/canary_ghpr_unit_test
/test pull-common-test
/test pull-e2e-test
/test pull-integration-common-test
/test pull-integration-copr-test
/test pull-integration-jdbc-test
/test pull-integration-mysql-test
/test pull-integration-nodejs-test
/test pull-sqllogic-test
/test pull-tiflash-test

Use /test all to run the following jobs that were automatically triggered:

pingcap/tidb/ghpr_build
pingcap/tidb/ghpr_check
pingcap/tidb/ghpr_check2
pingcap/tidb/ghpr_mysql_test
pingcap/tidb/ghpr_unit_test
pingcap/tidb/pull_integration_ddl_test
pingcap/tidb/pull_mysql_client_test

In response to this:

/test tidb_parser_test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Apr 29 '24 10:04 ti-chi-bot[bot]

@ystaticy: The specified target(s) for /test were not found. The following commands are available to trigger required jobs:

/test fast_test_tiprow
/test tidb_parser_test

Use /test all to run all jobs.

In response to this:

/test tidb_parser_test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Apr 29 '24 10:04 tiprow[bot]

/test all

Apr 29 '24 10:04 ystaticy

/test check-dev2

Apr 30 '24 02:04 ystaticy

@ystaticy: The specified target(s) for /test were not found. The following commands are available to trigger required jobs:

/test fast_test_tiprow
/test tidb_parser_test

Use /test all to run all jobs.

In response to this:

/test check-dev2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Apr 30 '24 02:04 tiprow[bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: AmoebaProtozoa Once this PR has been reviewed and has the lgtm label, please assign wjhuang2016 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS
pkg/domain/OWNERS

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

May 06 '24 04:05 ti-chi-bot[bot]

@AmoebaProtozoa: adding LGTM is restricted to approvers and reviewers in OWNERS files.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

May 06 '24 04:05 ti-chi-bot[bot]

/test check-dev

May 06 '24 05:05 ystaticy

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Aug 12 '24 05:08 ti-chi-bot[bot]

@ystaticy: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-lightning-integration-test	c543d869edb7da7d964767497b030f2386ea4c70	link	true	`/test pull-lightning-integration-test`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Sep 04 '24 06:09 ti-chi-bot[bot]

tidb tidb copied to clipboard

[DNM]GC: Support keyspace level GC

What problem does this PR solve?

What changed and how does it work?

Check List

Release note

Codecov Report

tidb
tidb copied to clipboard