lease: keep alive Leadership immediately after finish the election of leader
What problem does this PR solve?
Issue Number: Ref #7251
What is changed and how does it work?
// granted 3s
[2024/01/15 02:55:44.603 +00:00] [INFO] [server.go:1657] ["start to campaign PD leader"] [campaign-leader-name=tc-pd-0]
[2024/01/15 02:55:44.604 +00:00] [INFO] [lease.go:66] ["lease granted"] [lease-id=6177967866903092454] [lease-timeout=3] [purpose="leader election"]
// commit too slow
[2024/01/15 02:55:45.092 +00:00] [WARN] [etcdutil.go:152] ["kv gets too slow"] [request-key=/pd/7324144488088872732/member/5619078945150009647/leader_priority] [cost=2.764836133s] []
[2024/01/15 02:55:50.563 +00:00] [INFO] [etcdutil.go:314] ["update endpoints"] [num-change=3->2] [last-endpoints="[http://tc-pd-0.tc-pd-peer.endless-ha-test-tps-6030041-1-263.svc:2379,http://tc-pd-2.tc-pd-peer.endless-ha-test-tps-6030041-1-263.svc:2379,http://tc-pd-1.tc-pd-peer.endless-ha-test-tps-6030041-1-263.svc:2379]"] [endpoints="[http://tc-pd-0.tc-pd-peer.endless-ha-test-tps-6030041-1-263.svc:2379,http://tc-pd-2.tc-pd-peer.endless-ha-test-tps-6030041-1-263.svc:2379]"]
[2024/01/15 02:55:50.564 +00:00] [INFO] [etcdutil.go:314] ["update endpoints"] [num-change=3->2] [last-endpoints="[http://tc-pd-2.tc-pd-peer.endless-ha-test-tps-6030041-1-263.svc:2379,http://tc-pd-0.tc-pd-peer.endless-ha-test-tps-6030041-1-263.svc:2379,http://tc-pd-1.tc-pd-peer.endless-ha-test-tps-6030041-1-263.svc:2379]"] [endpoints="[http://tc-pd-2.tc-pd-peer.endless-ha-test-tps-6030041-1-263.svc:2379,http://tc-pd-0.tc-pd-peer.endless-ha-test-tps-6030041-1-263.svc:2379]"]
[2024/01/15 02:55:51.020 +00:00] [WARN] [etcd_kv.go:174] ["txn runs too slow"] [response="{\"header\":{\"cluster_id\":10740517942856960270,\"member_id\":17574458819109638910,\"revision\":1502,\"raft_term\":16},\"succeeded\":true,\"responses\":[{\"Response\":{\"ResponsePut\":{\"header\":{\"revision\":1502}}}}]}"] [cost=6.415850764s] []
// `02:55:51` pass 7s than granted
[2024/01/15 02:55:51.020 +00:00] [INFO] [leadership.go:181] ["check campaign resp"] [resp="{\"header\":{\"cluster_id\":10740517942856960270,\"member_id\":17574458819109638910,\"revision\":1502,\"raft_term\":16},\"succeeded\":true,\"responses\":[{\"Response\":{\"ResponsePut\":{\"header\":{\"revision\":1502}}}}]}"]
[2024/01/15 02:55:51.020 +00:00] [INFO] [leadership.go:190] ["write leaderData to leaderPath ok"] [leader-key=/pd/7324144488088872732/leader] [purpose="leader election"]
[2024/01/15 02:55:51.020 +00:00] [INFO] [server.go:1683] ["campaign PD leader ok"] [campaign-leader-name=tc-pd-0]
[2024/01/15 02:55:51.020 +00:00] [INFO] [server.go:1691] ["initializing the global TSO allocator"]
[2024/01/15 02:55:51.020 +00:00] [INFO] [tso.go:160] ["start to sync timestamp"] []
// check lease failed
[2024/01/15 02:55:51.020 +00:00] [INFO] [lease.go:155] ["start lease keep alive worker"] [interval=1s] [purpose="leader election"]
[2024/01/15 02:55:51.020 +00:00] [WARN] [lease.go:170] ["lease keep alive failed"] [purpose="leader election"] [start=2024/01/15 02:55:51.020 +00:00] [error="etcdserver: requested lease not found"]
...
[2024/01/15 02:55:51.038 +00:00] [INFO] [lease.go:187] ["stop lease keep alive worker"] [purpose="leader election"]
[2024/01/15 02:55:51.039 +00:00] [INFO] [server.go:1657] ["start to campaign PD leader"] [campaign-leader-name=tc-pd-0]
...
Check List
Tests
- Unit test
- Integration test
- Manual test (add detailed scripts or steps below)
Release note
None.
[REVIEW NOTIFICATION]
This pull request has not been approved.
To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.
The full list of commands accepted by this bot can be found here.
Reviewer can indicate their review by submitting an approval review. Reviewer can cancel approval by submitting a request changes review.
Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all
Codecov Report
Merging #7707 (4292223) into master (cd0ffba) will decrease coverage by
0.18%. Report is 1 commits behind head on master. The diff coverage is96.66%.
Additional details and impacted files
@@ Coverage Diff @@
## master #7707 +/- ##
==========================================
- Coverage 73.69% 73.51% -0.18%
==========================================
Files 432 432
Lines 47805 47787 -18
==========================================
- Hits 35229 35132 -97
- Misses 9572 9637 +65
- Partials 3004 3018 +14
| Flag | Coverage Δ | |
|---|---|---|
| unittests | 73.51% <96.66%> (-0.18%) |
:arrow_down: |
Flags with carried forward coverage won't be shown. Click here to find out more.
Do we still need this pr? Will it need to be picked to 7.5?
Do we still need this pr? Will it need to be picked to 7.5?
I don't think this pr is needed for now, Let's close it for now.