origin
origin copied to clipboard
openshift
NO-JIRA: We should always run tests where we have no opinion on their gate
When a test is added, that contains [OCPFeatureGate:, we expect that the gate already exists in openshift/api and we therefore have an opinion about the gate as to whether it is enabled or disabled.
In all cases, we render every known gate as either enabled or disabled, so every gate has an opinion.
However, when we remove gates, we no longer have an opinion, and the current code disables the tests.
Generally, removing the gate means the feature was promoted to default in a previous release, and therefore the tests should continue to run.
In some cases, the gate may be removed without ever graduating. This is rare, and in these cases we would start running their tests with the feature removed, which would fail, and we would find out pretty quickly.
This seems like the lesser of two evils, the alternative is we just stop running tests and are oblivious to this
@JoelSpeed: This pull request explicitly references no jira issue.
In response to this:
When a test is added, that contains
[OCPFeatureGate:, we expect that the gate already exists in openshift/api and we therefore have an opinion about the gate as to whether it is enabled or disabled.In all cases, we render every known gate as either enabled or disabled, so every gate has an opinion.
However, when we remove gates, we no longer have an opinion, and the current code disables the tests.
Generally, removing the gate means the feature was promoted to default in a previous release, and therefore the tests should continue to run.
In some cases, the gate may be removed without ever graduating. This is rare, and in these cases we would start running their tests with the feature removed, which would fail, and we would find out pretty quickly.
This seems like the lesser of two evils, the alternative is we just stop running tests and are oblivious to this
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.
@stbenjam: trigger 11 job(s) of type blocking for the nightly release of OCP 4.19
- periodic-ci-openshift-release-master-ci-4.19-e2e-aws-upgrade-ovn-single-node
- periodic-ci-openshift-release-master-nightly-4.19-e2e-aws-ovn-upgrade-fips
- periodic-ci-openshift-release-master-ci-4.19-e2e-azure-ovn-upgrade
- periodic-ci-openshift-release-master-ci-4.19-upgrade-from-stable-4.18-e2e-gcp-ovn-rt-upgrade
- periodic-ci-openshift-hypershift-release-4.19-periodics-e2e-aws-ovn-conformance
- periodic-ci-openshift-release-master-nightly-4.19-e2e-aws-ovn-serial
- periodic-ci-openshift-release-master-ci-4.19-e2e-aws-ovn-techpreview
- periodic-ci-openshift-release-master-ci-4.19-e2e-aws-ovn-techpreview-serial
- periodic-ci-openshift-release-master-nightly-4.19-fips-payload-scan
- periodic-ci-openshift-release-master-nightly-4.19-e2e-metal-ipi-ovn-bm
- periodic-ci-openshift-release-master-nightly-4.19-e2e-metal-ipi-ovn-ipv6
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/9a9a3730-4220-11f0-96f2-b19db0adfc1b-0
![openshift-ci[bot] avatar](https://avatars.githubusercontent.com/in/91280?v=4 "Published by a pub.dev verified publisher"){kind=small}
@stbenjam: aborted active payload jobs for pull request openshift/origin#29828
@stbenjam: trigger 11 job(s) of type blocking for the nightly release of OCP 4.20
- periodic-ci-openshift-release-master-ci-4.20-e2e-aws-upgrade-ovn-single-node
- periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-upgrade-fips
- periodic-ci-openshift-release-master-ci-4.20-e2e-azure-ovn-upgrade
- periodic-ci-openshift-release-master-ci-4.20-upgrade-from-stable-4.19-e2e-gcp-ovn-rt-upgrade
- periodic-ci-openshift-hypershift-release-4.20-periodics-e2e-aws-ovn-conformance
- periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-serial
- periodic-ci-openshift-release-master-ci-4.20-e2e-aws-ovn-techpreview
- periodic-ci-openshift-release-master-ci-4.20-e2e-aws-ovn-techpreview-serial
- periodic-ci-openshift-release-master-nightly-4.20-fips-payload-scan
- periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-ovn-bm
- periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-ovn-ipv6
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/b150bad0-4220-11f0-8332-024f4c65541c-0
@JoelSpeed: trigger 11 job(s) of type blocking for the nightly release of OCP 4.20
- periodic-ci-openshift-release-master-ci-4.20-e2e-aws-upgrade-ovn-single-node
- periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-upgrade-fips
- periodic-ci-openshift-release-master-ci-4.20-e2e-azure-ovn-upgrade
- periodic-ci-openshift-release-master-ci-4.20-upgrade-from-stable-4.19-e2e-gcp-ovn-rt-upgrade
- periodic-ci-openshift-hypershift-release-4.20-periodics-e2e-aws-ovn-conformance
- periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-serial
- periodic-ci-openshift-release-master-ci-4.20-e2e-aws-ovn-techpreview
- periodic-ci-openshift-release-master-ci-4.20-e2e-aws-ovn-techpreview-serial
- periodic-ci-openshift-release-master-nightly-4.20-fips-payload-scan
- periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-ovn-bm
- periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-ovn-ipv6
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/155ef1a0-45e0-11f0-9381-746e51c5ad0c-0
@JoelSpeed: trigger 11 job(s) of type blocking for the nightly release of OCP 4.20
- periodic-ci-openshift-release-master-ci-4.20-e2e-aws-upgrade-ovn-single-node
- periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-upgrade-fips
- periodic-ci-openshift-release-master-ci-4.20-e2e-azure-ovn-upgrade
- periodic-ci-openshift-release-master-ci-4.20-upgrade-from-stable-4.19-e2e-gcp-ovn-rt-upgrade
- periodic-ci-openshift-hypershift-release-4.20-periodics-e2e-aws-ovn-conformance
- periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-serial
- periodic-ci-openshift-release-master-ci-4.20-e2e-aws-ovn-techpreview
- periodic-ci-openshift-release-master-ci-4.20-e2e-aws-ovn-techpreview-serial
- periodic-ci-openshift-release-master-nightly-4.20-fips-payload-scan
- periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-ovn-bm
- periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-ovn-ipv6
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/1afe36b0-4a8c-11f0-89cc-cff19ee88b59-0
I've reviewed the output of the payload jobs this morning and checked the JUnit from several of the jobs. As far as i can tell this is still enabling the same set of jobs. I found some discrepancies between the JUnits but these can be attributed to the fact that a number of the /payload jobs used a CI build which included various test renames, where the actual nightlies use the nightly images and didn't have those test renames yet.
I'm confident in this PR and believe we should move forward.
/retest
[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: JoelSpeed, stbenjam
The full list of commands accepted by this bot can be found here.
The pull request process is described here
- ~~OWNERS~~ [stbenjam]
Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment
/retest-required
Remaining retests: 0 against base HEAD 140c6726a606a876153805cbcfe4b4b8c3dedf91 and 2 for PR HEAD 89e5a3ab2f2ba32ab2ebfb34230e0f51a996e00a in total
/retest-required
Remaining retests: 0 against base HEAD 140c6726a606a876153805cbcfe4b4b8c3dedf91 and 2 for PR HEAD 89e5a3ab2f2ba32ab2ebfb34230e0f51a996e00a in total
/retest-required
Remaining retests: 0 against base HEAD 140c6726a606a876153805cbcfe4b4b8c3dedf91 and 2 for PR HEAD 89e5a3ab2f2ba32ab2ebfb34230e0f51a996e00a in total
/retest-required
Remaining retests: 0 against base HEAD bea636f988a7f190e5e68609f1aeea531d7987fa and 1 for PR HEAD 89e5a3ab2f2ba32ab2ebfb34230e0f51a996e00a in total
/retest-required
Remaining retests: 0 against base HEAD bea636f988a7f190e5e68609f1aeea531d7987fa and 2 for PR HEAD 89e5a3ab2f2ba32ab2ebfb34230e0f51a996e00a in total
/retest-required
Remaining retests: 0 against base HEAD 14f46467649637aa71a57075d7094512f610f7b9 and 1 for PR HEAD 89e5a3ab2f2ba32ab2ebfb34230e0f51a996e00a in total
/retest-required
Remaining retests: 0 against base HEAD 044b48463cca5ecb4a52f3103819bb6a49eb4fd5 and 0 for PR HEAD 89e5a3ab2f2ba32ab2ebfb34230e0f51a996e00a in total
/hold
Revision 89e5a3ab2f2ba32ab2ebfb34230e0f51a996e00a was retested 3 times: holding