openshift
OCPBUGS-60853: cert tests: mark TLS registry test as informing
Reapply the changes from PR #29074 that were previously reverted, and mark both certificate tests as informing so failures don't block CI jobs.
Changes
Commit 1: Reapply PR #29074 (was reverted in #30358)
- Changes
testresult.Flakef()tog.Fail()for certificate tests - Removes TODO comments about making tests required
- Original PR: #29074
- This commit brings back the proper failure reporting mechanism
Commit 2: Mark TLS certificate tests as informing
- Added import for
github.com/openshift-eng/openshift-tests-extension/pkg/ginkgo - Added
ote.Informing()to both certificate tests:- "all tls artifacts must be registered"
- "all registered tls artifacts must have no metadata violation regressions"
Behavior
After these changes, both certificate tests will:
- Continue to run in blocking CI jobs
- Use
g.Fail()for proper failure reporting - Have
ote.Informing()label → Non-blocking (failures won't fail CI) - Gather data about TLS artifacts and metadata violations without blocking PRs
Benefits
- Tests provide proper failure reports with
g.Fail() - Tests are marked as informing, so failures are recorded but don't block development
- We can gather data and stabilize the tests while work continues on certificate registration and metadata compliance
- Once tests are stable, the
ote.Informing()label can be removed to make them blocking again
Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.
For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.
This repository is configured in: automatic mode
Scheduling required tests: /test e2e-aws-csi /test e2e-aws-ovn-fips /test e2e-aws-ovn-microshift /test e2e-aws-ovn-microshift-serial /test e2e-aws-ovn-serial-1of2 /test e2e-aws-ovn-serial-2of2 /test e2e-gcp-csi /test e2e-gcp-ovn /test e2e-gcp-ovn-upgrade /test e2e-metal-ipi-ovn-ipv6 /test e2e-vsphere-ovn /test e2e-vsphere-ovn-upi
@wangke19: This pull request references Jira Issue OCPBUGS-60853, which is invalid:
- expected the bug to target either version "4.21." or "openshift-4.21.", but it targets "4.20.z" instead
Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.
The bug has been updated to refer to the pull request using the external bug tracker.
In response to this:
Reapply the changes from PR #29074 that were previously reverted, and mark both certificate tests as informing so failures don't block CI jobs.
Changes
Commit 1: Reapply PR #29074 (was reverted in #30358)
- Changes
testresult.Flakef()tog.Fail()for certificate tests- Removes TODO comments about making tests required
- Original PR: #29074
- This commit brings back the proper failure reporting mechanism
Commit 2: Mark TLS certificate tests as informing
- Added import for
github.com/openshift-eng/openshift-tests-extension/pkg/ginkgo- Added
ote.Informing()to both certificate tests:- "all tls artifacts must be registered"
- "all registered tls artifacts must have no metadata violation regressions"
Behavior
After these changes, both certificate tests will:
- Continue to run in blocking CI jobs
- Use
g.Fail()for proper failure reporting- Have
ote.Informing()label → Non-blocking (failures won't fail CI)- Gather data about TLS artifacts and metadata violations without blocking PRs
Benefits
- Tests provide proper failure reports with
g.Fail()- Tests are marked as informing, so failures are recorded but don't block development
- We can gather data and stabilize the tests while work continues on certificate registration and metadata compliance
- Once tests are stable, the
ote.Informing()label can be removed to make them blocking again
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.
Scheduling required tests: /test e2e-aws-csi /test e2e-aws-ovn-fips /test e2e-aws-ovn-microshift /test e2e-aws-ovn-microshift-serial /test e2e-aws-ovn-serial-1of2 /test e2e-aws-ovn-serial-2of2 /test e2e-gcp-csi /test e2e-gcp-ovn /test e2e-gcp-ovn-upgrade /test e2e-metal-ipi-ovn-ipv6 /test e2e-vsphere-ovn /test e2e-vsphere-ovn-upi
@wangke19: This pull request references Jira Issue OCPBUGS-60853, which is valid.
3 validation(s) were run on this bug
- bug is open, matching expected state (open)
- bug target version (4.21.0) matches configured target version for branch (4.21.0)
- bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
Requesting review from QA contact: /cc @wangke19
In response to this:
/jira refresh
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.
@openshift-ci-robot: GitHub didn't allow me to request PR reviews from the following users: wangke19.
Note that only openshift members and repo collaborators can review this PR, and authors cannot review their own PRs.
In response to this:
@wangke19: This pull request references Jira Issue OCPBUGS-60853, which is valid.
3 validation(s) were run on this bug
- bug is open, matching expected state (open)
- bug target version (4.21.0) matches configured target version for branch (4.21.0)
- bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
Requesting review from QA contact: /cc @wangke19
In response to this:
/jira refresh
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
![openshift-ci[bot] avatar](https://avatars.githubusercontent.com/in/91280?v=4 "Published by a pub.dev verified publisher"){kind=small}
@wangke19: This pull request references Jira Issue OCPBUGS-60853, which is invalid:
- expected the bug to target either version "4.22." or "openshift-4.22.", but it targets "4.21.0" instead
Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.
In response to this:
Reapply the changes from PR #29074 that were previously reverted, mark both certificate tests as informing so failures don't block CI jobs, and skip the tests on ROSA clusters.
Changes
Commit 1: Reapply PR #29074 (was reverted in #30358)
- Changes
testresult.Flakef()tog.Fail()for certificate tests- Removes TODO comments about making tests required
- Original PR: #29074
- This commit brings back the proper failure reporting mechanism
Commit 2: Mark TLS certificate tests as informing
- Added import for
github.com/openshift-eng/openshift-tests-extension/pkg/ginkgo- Added
ote.Informing()to both certificate tests:- "all tls artifacts must be registered"
- "all registered tls artifacts must have no metadata violation regressions"
Commit 3: Skip TLS certificate tests on ROSA clusters
- Added
IsRosaCluster()function totest/extended/util/framework.go- Skip certificate tests on ROSA clusters (similar to MicroShift and Hypershift)
- ROSA clusters do not auto-collect TLS certificates the same way as standard OpenShift clusters
Behavior
After these changes:
On standard OpenShift clusters:
- Both certificate tests will continue to run in blocking CI jobs
- Tests use
g.Fail()for proper failure reporting- Tests have
ote.Informing()label → Non-blocking (failures won't fail CI)- Gather data about TLS artifacts and metadata violations without blocking PRs
On ROSA, MicroShift, and Hypershift clusters:
- Tests are skipped (these platforms don't auto-collect TLS the same way)
Benefits
- Tests provide proper failure reports with
g.Fail()- Tests are marked as informing, so failures are recorded but don't block development
- We can gather data and stabilize the tests while work continues on certificate registration and metadata compliance
- Avoid running tests on platforms that don't support TLS auto-collection
- Once tests are stable, the
ote.Informing()label can be removed to make them blocking again
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.
[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: wangke19 Once this PR has been reviewed and has the lgtm label, please assign dennisperiquet for approval. For more information see the Code Review Process.
The full list of commands accepted by this bot can be found here.
Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment
Scheduling required tests: /test e2e-aws-csi /test e2e-aws-ovn-fips /test e2e-aws-ovn-microshift /test e2e-aws-ovn-microshift-serial /test e2e-aws-ovn-serial-1of2 /test e2e-aws-ovn-serial-2of2 /test e2e-gcp-csi /test e2e-gcp-ovn /test e2e-gcp-ovn-upgrade /test e2e-metal-ipi-ovn-ipv6 /test e2e-vsphere-ovn /test e2e-vsphere-ovn-upi
@wangke19: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:
| Test name | Commit | Details | Required | Rerun command |
|---|---|---|---|---|
| ci/prow/e2e-metal-ipi-ovn-ipv6 | f7b77f2df412754d6188d70a355c81ebb87fe1ce | link | true | /test e2e-metal-ipi-ovn-ipv6 |
Full PR test history. Your PR dashboard.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.