Add sleep before shutdown backup container

[wxiaomou]

What problem does this PR solve?

Add sleep before shutdown backup container

What is changed and how does it work?

Code changes

• [ ] Has Go code change
• [ ] Has CI related scripts change

Tests

• [ ] Unit test
• [ ] E2E test
• [ ] Manual test
• [ ] No code

Side effects

• [ ] Breaking backward compatibility
• [ ] Other side effects:

Related changes

• [ ] Need to cherry-pick to the release branch
• [ ] Need to update the documentation

Release Notes

Please refer to Release Notes Language Style Guide before writing the release note.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 61.48%. Comparing base (89523ac) to head (ea9e867).

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #5454   +/-   ##
=======================================
  Coverage   61.47%   61.48%           
=======================================
  Files         235      235           
  Lines       30337    30337           
=======================================
+ Hits        18650    18652    +2     
+ Misses       9816     9815    -1     
+ Partials     1871     1870    -1     

Flag	Coverage Δ
unittest	61.48% <ø> (+<0.01%)	:arrow_up:

[BornChanger]

/retest

[BornChanger]

/retest

[wxiaomou]

@WangLe1321 our logging sidecar will flash log by batch, the timeout is set to 5s, if we reduce the sleep to 10s does it work for you? Current cleanup pod disappear within 5s, it makes it very hard to discover it.

[WangLe1321]

@WangLe1321 our logging sidecar will flash log by batch, the timeout is set to 5s, if we reduce the sleep to 10s does it work for you? Current cleanup pod disappear within 5s, it makes it very hard to discover it.

@wxiaomou you can only sleep in the clean command. I don't want to influence all the backup pods.

[nkg-]

@WangLe1321 our logging sidecar will flash log by batch, the timeout is set to 5s, if we reduce the sleep to 10s does it work for you? Current cleanup pod disappear within 5s, it makes it very hard to discover it.

@wxiaomou you can only sleep in the clean command. I don't want to influence all the backup pods.

Backup pods have the same need right. Some time to publish logs externally. Now I'm not sure how much time is sufficient. @wxiaomou : Do we need 30 seconds. Or is 10 seconds sufficient.

[wxiaomou]

@nkg- 10 second should be good @WangLe1321 I can remove the sleep after touch /tmp/pod/main-terminated but we need 10 second sleep before it to give our logging sidecar enough time to upload complete logs. This applied to all br pods.

[WangLe1321]

/run-pull-e2e-kind-br

[BornChanger]

/merge

[ti-chi-bot[bot]]

@BornChanger: We have migrated to builtin LGTM and approve plugins for reviewing.

Please use /approve when you want approve this pull request.

The changes announcement: LGTM plugin changes

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

[ti-chi-bot[bot]]

@wxiaomou: Your PR was out of date, I have automatically updated it for you.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

[BornChanger]

/cherry-pick release-1.5

[ti-chi-bot]

@BornChanger: once the present PR merges, I will cherry-pick it on top of release-1.5 in the new PR and assign it to you.

In response to this:

/cherry-pick release-1.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

[ti-chi-bot[bot]]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: BornChanger, csuzhangxc, WangLe1321

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• ~~OWNERS~~ [BornChanger,WangLe1321,csuzhangxc]

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot[bot]]

[LGTM Timeline notifier]

Timeline:

• 2023-12-13 10:48:19.402904673 +0000 UTC m=+439590.440131582: :ballot_box_with_check: agreed by BornChanger.
• 2023-12-13 11:17:41.864411345 +0000 UTC m=+441352.901638271: :heavy_multiplication_x::repeat: reset by WangLe1321.
• 2024-04-03 06:41:54.502207765 +0000 UTC m=+426176.029748311: :ballot_box_with_check: agreed by WangLe1321.
• 2024-04-03 08:09:26.342161239 +0000 UTC m=+431427.869701777: :heavy_multiplication_x::repeat: reset by ti-chi-bot[bot].
• 2024-04-03 08:32:46.10810059 +0000 UTC m=+432827.635641131: :ballot_box_with_check: agreed by csuzhangxc.

[BornChanger]

/rerun

[BornChanger]

/run-pull-e2e-kind-br

[ti-chi-bot]

@BornChanger: new pull request created to branch release-1.5: #5606.

In response to this:

/cherry-pick release-1.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.