origin icon indicating copy to clipboard operation
origin copied to clipboard

Bug 1885322: Increase OVN upgrade timeout by 15m

Open DavidHurta opened this issue 3 years ago • 27 comments
trafficstars

Cherry-picking https://github.com/openshift/origin/pull/25977, https://github.com/openshift/origin/pull/26207, https://github.com/openshift/origin/pull/26202, https://github.com/openshift/origin/pull/26324, and https://github.com/openshift/origin/pull/26327 to backport the fix regarding the bug 1885322 back to release-4.7.

DavidHurta avatar Mar 03 '22 12:03 DavidHurta

@Davoska: This pull request references Bugzilla bug 1885322, which is invalid:

  • expected the bug to target the "4.7.z" release, but it targets "---" instead
  • expected dependent Bugzilla bug 1879099 to target a release in 4.8.0, 4.8.z, but it targets "4.7.0" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Bug 1885322: Increase OVN upgrade timeout by 15m

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci[bot] avatar Mar 03 '22 12:03 openshift-ci[bot]

/bugzilla refresh

DavidHurta avatar Mar 03 '22 13:03 DavidHurta

@Davoska: This pull request references Bugzilla bug 1885322, which is invalid:

  • expected dependent Bugzilla bug 1879099 to target a release in 4.8.0, 4.8.z, but it targets "4.7.0" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci[bot] avatar Mar 03 '22 13:03 openshift-ci[bot]

/bugzilla refresh

DavidHurta avatar Mar 03 '22 13:03 DavidHurta

@Davoska: This pull request references Bugzilla bug 1885322, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

6 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.7.z) matches configured target release for branch (4.7.z)
  • bug is in the state NEW, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
  • dependent bug Bugzilla bug 1929650 is in the state CLOSED (ERRATA), which is one of the valid states (VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), CLOSED (CURRENTRELEASE))
  • dependent Bugzilla bug 1929650 targets the "4.8.z" release, which is one of the valid target releases: 4.8.0, 4.8.z
  • bug has dependents

Requesting review from QA contact: /cc @jianlinliu

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci[bot] avatar Mar 03 '22 13:03 openshift-ci[bot]

/hold ~~I need to double-check the contents of the commits.~~ Fixing the Pull Request - I have backported too much code.

DavidHurta avatar Mar 03 '22 14:03 DavidHurta

@Davoska: This pull request references Bugzilla bug 1885322, which is valid.

6 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.7.z) matches configured target release for branch (4.7.z)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
  • dependent bug Bugzilla bug 1929650 is in the state CLOSED (ERRATA), which is one of the valid states (VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), CLOSED (CURRENTRELEASE))
  • dependent Bugzilla bug 1929650 targets the "4.8.z" release, which is one of the valid target releases: 4.8.0, 4.8.z
  • bug has dependents

Requesting review from QA contact: /cc @jianlinliu

In response to this:

Bug 1885322: Increase OVN upgrade timeout by 15m

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci[bot] avatar Mar 03 '22 19:03 openshift-ci[bot]

/unhold

DavidHurta avatar Mar 03 '22 19:03 DavidHurta

$ cherry-pick-diff origin/release-4.7..origin/pr/26878 origin/release-4.8
e7456a82fd -> 6ea3f7c74d test/e2e/upgrade/upgrade: Unify duration-overshoot test-case name
785fe46c2b -> bdf1a8b0ba Bug 1942164: Fix time calc ordering for upgrades
834623fab6 -> 0ecd19ad3d Increase OVN upgrade timeout by 15m
a23e3eacd8 -> 1b35a9f313 test/e2e/upgrade: Relax 'too long' soft timeout for rollback jobs

e7456a82fd -> 6ea3f7c74d test/e2e/upgrade/upgrade: Unify duration-overshoot test-case name
...very clean, just some line number offsets...

785fe46c2b -> bdf1a8b0ba Bug 1942164: Fix time calc ordering for upgrades
...very clean, just some line number offsets...

834623fab6 -> 0ecd19ad3d Increase OVN upgrade timeout by 15m
...a bit messy, because 834623fab6 was touching some code from #26219 (AWS delay) that hasn't been taken back to 4.7.  Seems like 0ecd19ad3d is rolling those two changes into one commit...

a23e3eacd8 -> 1b35a9f313 test/e2e/upgrade: Relax 'too long' soft timeout for rollback jobs
...pretty clean: line offsets and minor context changes...

I dunno if it's worth picking #26219 back in here too. Might make folks less worried that we'd missed something in manual conflict resolution. But I'm not all that picky about 4.7 update code, so I don't mind some risk of minor nits if we do accidentally land a manual-resolution mistake.

wking avatar Mar 04 '22 04:03 wking

/hold Resolving the https://github.com/openshift/origin/pull/26878#issuecomment-1058826990 discussion.

DavidHurta avatar Mar 04 '22 08:03 DavidHurta

$ cherry-pick-diff origin/release-4.7..origin/pr/26878 origin/release-4.8
e7456a82fd -> 6ea3f7c74d test/e2e/upgrade/upgrade: Unify duration-overshoot test-case name
785fe46c2b -> bdf1a8b0ba Bug 1942164: Fix time calc ordering for upgrades
834623fab6 -> 0ecd19ad3d Increase OVN upgrade timeout by 15m
a23e3eacd8 -> 1b35a9f313 test/e2e/upgrade: Relax 'too long' soft timeout for rollback jobs

e7456a82fd -> 6ea3f7c74d test/e2e/upgrade/upgrade: Unify duration-overshoot test-case name
...very clean, just some line number offsets...

785fe46c2b -> bdf1a8b0ba Bug 1942164: Fix time calc ordering for upgrades
...very clean, just some line number offsets...

834623fab6 -> 0ecd19ad3d Increase OVN upgrade timeout by 15m
...a bit messy, because 834623fab6 was touching some code from #26219 (AWS delay) that hasn't been taken back to 4.7.  Seems like 0ecd19ad3d is rolling those two changes into one commit...

a23e3eacd8 -> 1b35a9f313 test/e2e/upgrade: Relax 'too long' soft timeout for rollback jobs
...pretty clean: line offsets and minor context changes...

I dunno if it's worth picking #26219 back in here too. Might make folks less worried that we'd missed something in manual conflict resolution. But I'm not all that picky about 4.7 update code, so I don't mind some risk of minor nits if we do accidentally land a manual-resolution mistake.

Thank you for the feedback. I have removed the https://github.com/openshift/origin/pull/26219 from the commit as you suggested. Now, this pull request should only increase the OVN upgrade timeout by 15 minutes and add some minor changes to the code. But it will not add the AWS delay.

I have also cherry-picked the https://github.com/openshift/origin/pull/26207 to add the commit of the fix to the history of the release-4.7 branch.

I will keep the label do-not-merge/hold for the moment. Let me know if my changes were adequate or if I should change something. It's my first backport.

Edit (clarifying commits): New mapping [original -> new]: https://github.com/openshift/origin/pull/25977 https://github.com/openshift/origin/pull/25977/commits/a23e3eacd871663fefced5b8a9cb8972d7108fbb -> https://github.com/openshift/origin/pull/26878/commits/395a299f89b34a4ab1003629d6ddcfe83ff72ee6 Relaxing soft timeout for rollback jobs which is used in the OVN upgrade timeout by 15m fix, minor context changes, wording...

https://github.com/openshift/origin/pull/26207 https://github.com/openshift/origin/pull/26207/commits/3c5c821a747eb603c4ca77168361ee8a3ea077e7 -> https://github.com/openshift/origin/pull/26878/commits/abe8f4edaffcd31eaf218b3da7a4a04bdaa4a87e Clarifying failure message without the mentioning that upgrades on AWS should take longer since we are not backporting https://github.com/openshift/origin/pull/26219.

https://github.com/openshift/origin/pull/26202 https://github.com/openshift/origin/pull/26202/commits/5ec3836e8acdb3d7d12c9af74abaee500c5b9d53 -> https://github.com/openshift/origin/pull/26878/commits/ee6265149767418e49c6fa433bdd397f77a6af4f Increasing OVN upgrade timeout by 15m without backporting the https://github.com/openshift/origin/pull/26219.

https://github.com/openshift/origin/pull/26324 https://github.com/openshift/origin/pull/26324/commits/5c5627526ac00f7e9c66fa3d3def8b5a22fee341 -> https://github.com/openshift/origin/pull/26878/commits/9772ac4f3af56709a51672d5c3053fc957c7ce39 Fix to the Increase OVN upgrade timeout by 15m.

https://github.com/openshift/origin/pull/26327 https://github.com/openshift/origin/pull/26327/commits/bbb3a70f07f17cffb79a15db5b0509b2762a8da9 -> https://github.com/openshift/origin/pull/26878/commits/5502ec398840b1d4a32f967bb871c11b82482c2d Unifying duration-overshoot test-case names...

DavidHurta avatar Mar 04 '22 10:03 DavidHurta

/retest-required

DavidHurta avatar Mar 04 '22 12:03 DavidHurta

/retest-required

DavidHurta avatar Mar 07 '22 13:03 DavidHurta

/retest-required

DavidHurta avatar Mar 07 '22 15:03 DavidHurta

@Davoska: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

openshift-ci[bot] avatar Mar 07 '22 17:03 openshift-ci[bot]

/unhold Failing checks have passed, removing the do-not-merge/hold label.

DavidHurta avatar Mar 07 '22 23:03 DavidHurta

@Davoska: This pull request references Bugzilla bug 1885322, which is valid.

6 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.7.z) matches configured target release for branch (4.7.z)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
  • dependent bug Bugzilla bug 1929650 is in the state CLOSED (ERRATA), which is one of the valid states (VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), CLOSED (CURRENTRELEASE))
  • dependent Bugzilla bug 1929650 targets the "4.8.z" release, which is one of the valid target releases: 4.8.0, 4.8.z
  • bug has dependents

Requesting review from QA contact: /cc @jianlinliu

In response to this:

Bug 1885322: Increase OVN upgrade timeout by 15m

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci[bot] avatar Mar 07 '22 23:03 openshift-ci[bot]

/hold

@Davoska I can not confirm if https://github.com/openshift/origin/pull/26202 is also back ported with this PR. Can you confirm?

LalatenduMohanty avatar Mar 22 '22 20:03 LalatenduMohanty

@LalatenduMohanty, the https://github.com/openshift/origin/pull/26202 is also backported with this pull request in the https://github.com/openshift/origin/pull/26878/commits/ee6265149767418e49c6fa433bdd397f77a6af4f commit.

Although cherry-picking the original PR https://github.com/openshift/origin/pull/26202 would also bring changes from the https://github.com/openshift/origin/pull/26219 because the https://github.com/openshift/origin/pull/26202 modifies some of its code. I have manually resolved the conflict so that the changes regarding the https://github.com/openshift/origin/pull/26219 are not backported.

DavidHurta avatar Mar 30 '22 11:03 DavidHurta

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close. Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

openshift-bot avatar Jun 28 '22 12:06 openshift-bot

/remove-lifecycle stale

DavidHurta avatar Jul 25 '22 06:07 DavidHurta

I have updated the comment https://github.com/openshift/origin/pull/26878#issuecomment-1059052900 to clarify the backported commits.

DavidHurta avatar Jul 25 '22 11:07 DavidHurta

/hold cancel

LalatenduMohanty avatar Aug 05 '22 20:08 LalatenduMohanty

/assign @soltysh Hello, tagging regarding the approved and backport-risk-assessed labels :wave:

DavidHurta avatar Aug 08 '22 18:08 DavidHurta

/approve

LalatenduMohanty avatar Aug 08 '22 20:08 LalatenduMohanty

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Davoska, LalatenduMohanty Once this PR has been reviewed and has the lgtm label, please ask for approval from soltysh by writing /assign @soltysh in a comment. For more information see:The Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci[bot] avatar Aug 08 '22 20:08 openshift-ci[bot]

@soltysh, let me know if there are any complications with the pull request.

DavidHurta avatar Aug 24 '22 11:08 DavidHurta

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close. Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

openshift-bot avatar Nov 23 '22 01:11 openshift-bot

As the bug itself is closed due to the OCP 4.7 being EOL, closing the pull request.

/close

DavidHurta avatar Nov 23 '22 13:11 DavidHurta

@Davoska: Closed this PR.

In response to this:

As the bug itself is closed due to the OCP 4.7 being EOL, closing the pull request.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci[bot] avatar Nov 23 '22 13:11 openshift-ci[bot]