tidb-operator icon indicating copy to clipboard operation
tidb-operator copied to clipboard

opt: tikvStatefulSetIsUpgrading check against tikv sts UpdateRevision

Open ideascf opened this issue 1 year ago • 2 comments

What problem does this PR solve?

In some cases, the tikvMemberManager of tidb-operator may regard the tikv as upgrading when the corresponding STS is creating. For example, if the tidb-operator retrieved tikv STS before it has finished the first reconciliation and retrieved tikv pods after it reconciled, https://github.com/pingcap/tidb-operator/blob/master/pkg/manager/member/tikv_member_manager.go#L1070-L1072 will wrongly regard the tikv STS as upgrading. In this situation, an unexpected upgradation and leader eviction of the largest ordinal pod of this STS will be triggered, https://github.com/pingcap/tidb-operator/blob/master/pkg/manager/member/tikv_member_manager.go#L292-L296.

What is changed and how does it work?

Code changes

  • [ ] Has Go code change
  • [ ] Has CI related scripts change

Tests

  • [ ] Unit test
  • [ ] E2E test
  • [ ] Manual test
  • [ ] No code

Side effects

  • [ ] Breaking backward compatibility
  • [ ] Other side effects:

Related changes

  • [ ] Need to cherry-pick to the release branch
  • [ ] Need to update the documentation

Release Notes

Please refer to Release Notes Language Style Guide before writing the release note.


ideascf avatar Feb 21 '24 06:02 ideascf

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 61.47%. Comparing base (05631f8) to head (5b29b2e).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #5551   +/-   ##
=======================================
  Coverage   61.47%   61.47%           
=======================================
  Files         235      235           
  Lines       30336    30337    +1     
=======================================
+ Hits        18649    18650    +1     
  Misses       9816     9816           
  Partials     1871     1871           
Flag Coverage Δ
unittest 61.47% <100.00%> (+<0.01%) :arrow_up:

codecov-commenter avatar Feb 21 '24 06:02 codecov-commenter

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Once this PR has been reviewed and has the lgtm label, please assign z2665 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot[bot] avatar Apr 02 '24 08:04 ti-chi-bot[bot]

/run-pull-e2e-kind

csuzhangxc avatar Apr 02 '24 08:04 csuzhangxc

/cherry-pick release-1.5

csuzhangxc avatar Apr 02 '24 11:04 csuzhangxc

@csuzhangxc: new pull request created to branch release-1.5: #5603.

In response to this:

/cherry-pick release-1.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot avatar Apr 02 '24 11:04 ti-chi-bot