origin icon indicating copy to clipboard operation
origin copied to clipboard

OPRUN-3576: Add OLMv1 origin tests

Open tmshort opened this issue 1 year ago • 12 comments

This is the initial test. Only works on tech-preview. Checks for presence of OLMv1 CRDs.

tmshort avatar Oct 14 '24 19:10 tmshort

@tmshort: This pull request references OPRUN-3576 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.18.0" version, but no target version was set.

In response to this:

This is the initial test. Only works on tech-preview. Checks for presence of OLMv1 CRDs.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot avatar Oct 14 '24 19:10 openshift-ci-robot

Skipping CI for Draft Pull Request. If you want CI signal for your change, please convert it to an actual PR. You can still manually trigger a test run with /test all

openshift-ci[bot] avatar Oct 14 '24 19:10 openshift-ci[bot]

/retest

tmshort avatar Oct 16 '24 18:10 tmshort

/retest

tmshort avatar Oct 17 '24 13:10 tmshort

Since you were lucky enough to be randomly chosen by github... @deads2k @mike-nguyen, my tests only do something under tech-preview (until OLMv1 is out of tech preview), but a number prow jobs seem to be failing? Is this normal? How do I get this PR to progress?

tmshort avatar Oct 17 '24 13:10 tmshort

@wking @petr-muller are my go to origin test experts. Can you guys take a look?

LalatenduMohanty avatar Oct 17 '24 16:10 LalatenduMohanty

Job Failure Risk Analysis for sha: f15e4ed8686b7e1ed11c193d0b9a3f2aca0b3104

Job Name Failure Risk
pull-ci-openshift-origin-master-e2e-metal-ipi-ovn High
[sig-api-machinery] disruption/cache-kube-api connection/new should be available throughout the test
This test has passed 99.77% of 5687 runs on release 4.18 [Overall] in the last week.
---
[sig-api-machinery] disruption/openshift-api connection/new should be available throughout the test
This test has passed 99.86% of 5687 runs on release 4.18 [Overall] in the last week.
---
[sig-api-machinery] disruption/kube-api connection/new should be available throughout the test
This test has passed 99.75% of 5687 runs on release 4.18 [Overall] in the last week.
---
[sig-api-machinery] disruption/cache-oauth-api connection/new should be available throughout the test
This test has passed 99.86% of 5687 runs on release 4.18 [Overall] in the last week.

openshift-trt-bot avatar Oct 17 '24 16:10 openshift-trt-bot

/retest

tmshort avatar Oct 17 '24 17:10 tmshort

/override ci/prow/e2e-aws-ovn-ipsec-serial

Known broken.

dgoodwin avatar Oct 17 '24 18:10 dgoodwin

@dgoodwin: Overrode contexts on behalf of dgoodwin: ci/prow/e2e-aws-ovn-ipsec-serial

In response to this:

/override ci/prow/e2e-aws-ovn-ipsec-serial

Known broken.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci[bot] avatar Oct 17 '24 18:10 openshift-ci[bot]

Seems like a reasonable foray into OLMv1 origin tests, but I do wonder if this will really stand the test of time.

I would somewhat assume that this would be quickly obviated by origin tests that actually exercise the APIs that we expect to exist.

This is a first pass at getting something in. The plan is to actually do a happy-path, but I had been waiting on other work to complete before I could get happy-path working.

tmshort avatar Oct 21 '24 15:10 tmshort

/retest

tmshort avatar Oct 21 '24 21:10 tmshort

Job Failure Risk Analysis for sha: 64bdcd727c23eccbf66e4c0917897af503ba6d62

Job Name Failure Risk
pull-ci-openshift-origin-master-e2e-metal-ipi-ovn-kube-apiserver-rollout IncompleteTests
Tests for this run (101) are below the historical average (1155): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-metal-ipi-ovn-ipv6 IncompleteTests
Tests for this run (101) are below the historical average (2074): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-metal-ipi-ovn IncompleteTests
Tests for this run (101) are below the historical average (2242): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

openshift-trt-bot avatar Oct 22 '24 02:10 openshift-trt-bot

/retest

tmshort avatar Oct 22 '24 13:10 tmshort

/retest

tmshort avatar Oct 22 '24 17:10 tmshort

/retest

tmshort avatar Oct 23 '24 21:10 tmshort

/retest

tmshort avatar Oct 24 '24 02:10 tmshort

Job Failure Risk Analysis for sha: 09474cdf0bf9fc603554e21f25e82fd21499245f

Job Name Failure Risk
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-upgrade High
[sig-arch] events should not repeat pathologically for ns/openshift-kube-apiserver
This test has passed 99.13% of 230 runs on release 4.18 [Architecture:amd64 FeatureSet:default Installer:ipi Network:ovn NetworkStack:ipv4 Platform:aws SecurityMode:default Topology:single Upgrade:micro] in the last week.

openshift-trt-bot avatar Oct 24 '24 06:10 openshift-trt-bot

@deads2k @mike-nguyen looking for approval

tmshort avatar Oct 24 '24 13:10 tmshort

@tmshort Any reason in particular we're creating a new test/extended subfolder? IMO we could also just stick this in the existing olm folder, and then we wouldn't need to manage another OWNERS file...

dtfranz avatar Oct 24 '24 15:10 dtfranz

@tmshort Any reason in particular we're creating a new test/extended subfolder? IMO we could also just stick this in the existing olm folder, and then we wouldn't need to manage another OWNERS file...

To keep them separate. Makes it easier for later... but I get your point...

tmshort avatar Oct 25 '24 14:10 tmshort

/approve

tmshort avatar Oct 25 '24 15:10 tmshort

/approve

kevinrizza avatar Oct 25 '24 18:10 kevinrizza

/retest

tmshort avatar Oct 25 '24 18:10 tmshort

/retest-required

Remaining retests: 0 against base HEAD 4560b2be97bfb63bab49409640bf99fd2d04e769 and 2 for PR HEAD d787b509d3a527ea173e974d30cf3b2410c6d169 in total

openshift-ci-robot avatar Oct 25 '24 18:10 openshift-ci-robot

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dtfranz, kevinrizza, LalatenduMohanty, tmshort

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci[bot] avatar Oct 25 '24 21:10 openshift-ci[bot]

@tmshort: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ovn-ipsec-serial f15e4ed8686b7e1ed11c193d0b9a3f2aca0b3104 link false /test e2e-aws-ovn-ipsec-serial
ci/prow/e2e-aws-ovn-single-node-serial d787b509d3a527ea173e974d30cf3b2410c6d169 link false /test e2e-aws-ovn-single-node-serial

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci[bot] avatar Oct 25 '24 21:10 openshift-ci[bot]

Job Failure Risk Analysis for sha: d787b509d3a527ea173e974d30cf3b2410c6d169

Job Name Failure Risk
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-serial Medium
[sig-node] static pods should start after being created
This test has passed 84.93% of 73 runs on jobs ['periodic-ci-openshift-release-master-nightly-4.18-e2e-aws-ovn-single-node-serial'] in the last 14 days.

Open Bugs
Static pod controller pods sometimes fail to start [kube-controller-manager]
Static pod controller pods sometimes fail to start [etcd]

openshift-trt-bot avatar Oct 25 '24 21:10 openshift-trt-bot

[ART PR BUILD NOTIFIER]

Distgit: openshift-enterprise-tests This PR has been included in build openshift-enterprise-tests-container-v4.18.0-202410260640.p0.g877a6e3.assembly.stream.el9. All builds following this will include this PR.

openshift-bot avatar Oct 26 '24 07:10 openshift-bot