origin icon indicating copy to clipboard operation
origin copied to clipboard
verified publisher icon openshift

OCPQE-23184: Support external cloud authentication providers

Open sunzhaohua2 opened this issue 1 year ago • 58 comments

This will add test about pulling image from ECR/GCR/ACR. As we are not allowed to push an image to container registry in advance, based on discussion , in code we create registry and mirror image, after testing remove registry.

  • We will set up a minimal, publicly pullable image (for ease) somewhere outside of the CI cloud accounts
  • The test will create an ECR (or equivalent) repo, and mirror the image across
  • Test will setup cleanup to remove the repo
  • PCO pruner will be updated to clean repos if we miss it

Feature Epic https://issues.redhat.com/browse/OCPCLOUD-2379

sunzhaohua2 avatar May 07 '24 06:05 sunzhaohua2

@sunzhaohua2: This pull request references OCPCLOUD-2558 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.16.0" version, but no target version was set.

In response to this:

Local test result: AWS

$ ./openshift-tests run-test "[sig-cluster-lifecycle][Feature:KubeletAuthenticationProviders][Serial] KubeletAuthenticationProviders should should be able to pull images from ECR"
INFO[0000] Decoding provider                             clusterState="<nil>" discover=false dryRun=false func=DecodeProvider providerType=
 May  7 14:20:19.722: INFO: microshift-version configmap not found
 Running Suite: OpenShift e2e suite - /Users/sunzhaohua/go/src/github.com/openshift/origin
 =========================================================================================
 Random Seed: 1715062817 - will randomize all specs

 Will run 1 of 1 specs
 ------------------------------
 [sig-cluster-lifecycle][Feature:KubeletAuthenticationProviders][Serial] KubeletAuthenticationProviders should should be able to pull images from ECR
 github.com/openshift/origin/test/extended/kubelet_authentication_providers/kubelet_authentication_providers.go:17
   STEP: Creating a kubernetes client @ 05/07/24 14:20:20.62
 May  7 14:20:23.590: INFO: configPath is now "/var/folders/0m/7xwxpmks77n3dm5rr8x8g92r0000gn/T/configfile247212266"
 May  7 14:20:23.590: INFO: The user is now "e2e-test-kubelet-authentication-providers-t28tx-user"
 May  7 14:20:23.590: INFO: Creating project "e2e-test-kubelet-authentication-providers-t28tx"
 May  7 14:20:23.901: INFO: Waiting on permissions in project "e2e-test-kubelet-authentication-providers-t28tx" ...
 May  7 14:20:24.533: INFO: Waiting for ServiceAccount "default" to be provisioned...
 May  7 14:20:24.849: INFO: Waiting for ServiceAccount "deployer" to be provisioned...
 May  7 14:20:25.160: INFO: Waiting for ServiceAccount "builder" to be provisioned...
 May  7 14:20:25.469: INFO: Waiting for RoleBinding "system:image-pullers" to be provisioned...
 May  7 14:20:25.884: INFO: Waiting for RoleBinding "system:image-builders" to be provisioned...
 May  7 14:20:26.298: INFO: Waiting for RoleBinding "system:deployers" to be provisioned...
 May  7 14:20:27.177: INFO: Project "e2e-test-kubelet-authentication-providers-t28tx" has been fully provisioned.
 May  7 14:20:27.177: INFO: Running 'oc --kubeconfig=/Users/sunzhaohua/kubeconfig/aws/kubeconfig get infrastructure cluster -o=jsonpath={.status.platformStatus.type}'
 May  7 14:20:27.948: INFO: Running 'oc --kubeconfig=/Users/sunzhaohua/kubeconfig/aws/kubeconfig get infrastructure cluster -o=jsonpath={.status.platformStatus.aws.region}'
   STEP: Add the AmazonEC2ContainerRegistryReadOnly policy to the worker nodes @ 05/07/24 14:20:28.719
 May  7 14:20:28.720: INFO: Running 'oc --kubeconfig=/Users/sunzhaohua/kubeconfig/aws/kubeconfig get infrastructure cluster -o=jsonpath={.status.infrastructureName}'
 May  7 14:20:29.482: INFO: Running 'oc --kubeconfig=/Users/sunzhaohua/kubeconfig/aws/kubeconfig get secret/aws-creds -n kube-system -o json'
 May  7 14:20:30.239: INFO: Running 'oc --kubeconfig=/Users/sunzhaohua/kubeconfig/aws/kubeconfig get infrastructure cluster -o=jsonpath={.status.platformStatus.aws.region}'
   STEP: Create a new app using the image on ECR @ 05/07/24 14:20:32.184
 May  7 14:20:32.184: INFO: Running 'oc --kubeconfig=/Users/sunzhaohua/kubeconfig/aws/kubeconfig new-app --name=hello-ecr --image=301721915996.dkr.ecr.us-east-2.amazonaws.com/hello-ecr:latest --allow-missing-images -n e2e-test-kubelet-authentication-providers-t28tx'
 W0507 14:20:33.123096   62275 newapp.go:523] Could not find an image stream match for "301721915996.dkr.ecr.us-east-2.amazonaws.com/hello-ecr:latest". Make sure that a container image with that tag is available on the node for the deployment to succeed.
 --> Found container image 7af3297 (6 years old) from 301721915996.dkr.ecr.us-east-2.amazonaws.com for "301721915996.dkr.ecr.us-east-2.amazonaws.com/hello-ecr:latest"


 --> Creating resources ...
     deployment.apps "hello-ecr" created
     service "hello-ecr" created
 --> Success
     Application is not exposed. You can expose services to the outside world by executing one or more of the commands below:
      'oc expose service/hello-ecr' 
     Run 'oc status' to view your app.
   STEP: Wait the pod to be running @ 05/07/24 14:20:33.575
 May  7 14:20:48.282: INFO: Deleted {user.openshift.io/v1, Resource=users  e2e-test-kubelet-authentication-providers-t28tx-user}, err: <nil>
 May  7 14:20:48.495: INFO: Deleted {oauth.openshift.io/v1, Resource=oauthclients  e2e-client-e2e-test-kubelet-authentication-providers-t28tx}, err: <nil>
 May  7 14:20:48.707: INFO: Deleted {oauth.openshift.io/v1, Resource=oauthaccesstokens  sha256~YD4cCjytD1tSyKCjlOcCEjUjTURQsbFa6xubRN4Frvg}, err: <nil>
   STEP: Destroying namespace "e2e-test-kubelet-authentication-providers-t28tx" for this suite. @ 05/07/24 14:20:48.708
 • [28.306 seconds]
 ------------------------------

 Ran 1 of 1 Specs in 28.306 seconds

GCP

$ ./openshift-tests run-test "[sig-cluster-lifecycle][Feature:KubeletAuthenticationProviders][Serial] KubeletAuthenticationProviders should should be able to pull images from GCR"
INFO[0000] Decoding provider                             clusterState="<nil>" discover=false dryRun=false func=DecodeProvider providerType=
 May  7 14:18:15.334: INFO: Fetching cloud provider for "gce"
 I0507 14:18:15.338631   62120 gce.go:937] Using DefaultTokenSource &google.errWrappingTokenSource{src:(*oauth2.reuseTokenSource)(0xc001946ab0)}
 W0507 14:18:15.710738   62120 gce.go:491] No network name or URL specified.
 I0507 14:18:15.710827   62120 gce.go:512] managing multiple zones: [us-central1-a us-central1-b us-central1-c]
 May  7 14:18:15.930: INFO: microshift-version configmap not found
 Running Suite: OpenShift e2e suite - /Users/sunzhaohua/go/src/github.com/openshift/origin
 =========================================================================================
 Random Seed: 1715062684 - will randomize all specs

 Will run 1 of 1 specs
 ------------------------------
 [sig-cluster-lifecycle][Feature:KubeletAuthenticationProviders][Serial] KubeletAuthenticationProviders should should be able to pull images from GCR
 github.com/openshift/origin/test/extended/kubelet_authentication_providers/kubelet_authentication_providers.go:49
   STEP: Creating a kubernetes client @ 05/07/24 14:18:16.801
 May  7 14:18:31.986: INFO: configPath is now "/var/folders/0m/7xwxpmks77n3dm5rr8x8g92r0000gn/T/configfile2520542568"
 May  7 14:18:31.986: INFO: The user is now "e2e-test-kubelet-authentication-providers-vnrqj-user"
 May  7 14:18:31.986: INFO: Creating project "e2e-test-kubelet-authentication-providers-vnrqj"
 May  7 14:18:32.288: INFO: Waiting on permissions in project "e2e-test-kubelet-authentication-providers-vnrqj" ...
 May  7 14:18:33.042: INFO: Waiting for ServiceAccount "default" to be provisioned...
 May  7 14:18:33.358: INFO: Waiting for ServiceAccount "deployer" to be provisioned...
 May  7 14:18:33.678: INFO: Waiting for ServiceAccount "builder" to be provisioned...
 May  7 14:18:33.996: INFO: Waiting for RoleBinding "system:image-pullers" to be provisioned...
 May  7 14:18:34.681: INFO: Waiting for RoleBinding "system:image-builders" to be provisioned...
 May  7 14:18:35.111: INFO: Waiting for RoleBinding "system:deployers" to be provisioned...
 May  7 14:18:36.257: INFO: Project "e2e-test-kubelet-authentication-providers-vnrqj" has been fully provisioned.
 May  7 14:18:36.262: INFO: Running 'oc --kubeconfig=/Users/sunzhaohua/kubeconfig/gcp/kubeconfig get infrastructure cluster -o=jsonpath={.status.platformStatus.type}'
   STEP: Create a new app using the image on GCR @ 05/07/24 14:18:37.17
 May  7 14:18:37.170: INFO: Running 'oc --kubeconfig=/Users/sunzhaohua/kubeconfig/gcp/kubeconfig new-app --name=hello-gcr --image=gcr.io/openshift-qe/hello-gcr:latest --allow-missing-images -n e2e-test-kubelet-authentication-providers-vnrqj'
 W0507 14:18:38.294544   62139 newapp.go:523] Could not find an image stream match for "gcr.io/openshift-qe/hello-gcr:latest". Make sure that a container image with that tag is available on the node for the deployment to succeed.
 --> Found container image 7af3297 (6 years old) from gcr.io for "gcr.io/openshift-qe/hello-gcr:latest"


 --> Creating resources ...
     deployment.apps "hello-gcr" created
     service "hello-gcr" created
 --> Success
     Application is not exposed. You can expose services to the outside world by executing one or more of the commands below:
      'oc expose service/hello-gcr' 
     Run 'oc status' to view your app.
   STEP: Wait the pod to be running @ 05/07/24 14:18:38.88
 May  7 14:18:41.333: INFO: Deleted {user.openshift.io/v1, Resource=users  e2e-test-kubelet-authentication-providers-vnrqj-user}, err: <nil>
 May  7 14:18:41.555: INFO: Deleted {oauth.openshift.io/v1, Resource=oauthclients  e2e-client-e2e-test-kubelet-authentication-providers-vnrqj}, err: <nil>
 May  7 14:18:42.770: INFO: Deleted {oauth.openshift.io/v1, Resource=oauthaccesstokens  sha256~8j4UO-61tJBnoRFZaQV4AhmpXPhjN0qbxx3vP5YRGL8}, err: <nil>
   STEP: Destroying namespace "e2e-test-kubelet-authentication-providers-vnrqj" for this suite. @ 05/07/24 14:18:42.771
 • [26.408 seconds]
 ------------------------------

 Ran 1 of 1 Specs in 26.409 seconds
 SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped

Azure

$ ./openshift-tests run-test "[sig-cluster-lifecycle][Feature:KubeletAuthenticationProviders][Serial] KubeletAuthenticationProviders should should be able to pull images from ACR"
INFO[0000] Decoding provider                             clusterState="<nil>" discover=false dryRun=false func=DecodeProvider providerType=
 May  7 14:14:50.515: INFO: microshift-version configmap not found
 Running Suite: OpenShift e2e suite - /Users/sunzhaohua/go/src/github.com/openshift/origin
 =========================================================================================
 Random Seed: 1715062487 - will randomize all specs

 Will run 1 of 1 specs
 ------------------------------
 [sig-cluster-lifecycle][Feature:KubeletAuthenticationProviders][Serial] KubeletAuthenticationProviders should should be able to pull images from ACR
 github.com/openshift/origin/test/extended/kubelet_authentication_providers/kubelet_authentication_providers.go:64
   STEP: Creating a kubernetes client @ 05/07/24 14:14:51.418
 May  7 14:14:55.477: INFO: configPath is now "/var/folders/0m/7xwxpmks77n3dm5rr8x8g92r0000gn/T/configfile2343691754"
 May  7 14:14:55.477: INFO: The user is now "e2e-test-kubelet-authentication-providers-mpvxd-user"
 May  7 14:14:55.477: INFO: Creating project "e2e-test-kubelet-authentication-providers-mpvxd"
 May  7 14:14:55.872: INFO: Waiting on permissions in project "e2e-test-kubelet-authentication-providers-mpvxd" ...
 May  7 14:14:56.778: INFO: Waiting for ServiceAccount "default" to be provisioned...
 May  7 14:14:57.164: INFO: Waiting for ServiceAccount "deployer" to be provisioned...
 May  7 14:14:57.553: INFO: Waiting for ServiceAccount "builder" to be provisioned...
 May  7 14:14:57.940: INFO: Waiting for RoleBinding "system:image-pullers" to be provisioned...
 May  7 14:14:58.506: INFO: Waiting for RoleBinding "system:image-builders" to be provisioned...
 May  7 14:14:59.090: INFO: Waiting for RoleBinding "system:deployers" to be provisioned...
 May  7 14:15:00.284: INFO: Project "e2e-test-kubelet-authentication-providers-mpvxd" has been fully provisioned.
 May  7 14:15:00.289: INFO: Running 'oc --kubeconfig=/Users/sunzhaohua/kubeconfig/azure/kubeconfig get infrastructure cluster -o=jsonpath={.status.platformStatus.type}'
 May  7 14:15:01.786: INFO: Running 'oc --kubeconfig=/Users/sunzhaohua/kubeconfig/azure/kubeconfig get infrastructure cluster -o=jsonpath={.status.platformStatus.azure.cloudName}'
   STEP: Create RoleAssignments for resourcegroup @ 05/07/24 14:15:02.868
 May  7 14:15:02.868: INFO: Running 'oc --kubeconfig=/Users/sunzhaohua/kubeconfig/azure/kubeconfig get infrastructure cluster -o=jsonpath={.status.infrastructureName}'
 May  7 14:15:04.245: INFO: Running 'oc --kubeconfig=/Users/sunzhaohua/kubeconfig/azure/kubeconfig get secret/azure-credentials -n kube-system -o=jsonpath={.data}'
 May  7 14:15:05.233: INFO: Azure credentials successfully loaded.
 May  7 14:15:13.478: INFO: Role assignment created: 53b8f551-140d-438c-ba67-43fe172d538d
   STEP: Create a new app using the image on ACR @ 05/07/24 14:15:13.478
 May  7 14:15:13.478: INFO: Running 'oc --kubeconfig=/Users/sunzhaohua/kubeconfig/azure/kubeconfig new-app --name=hello-acr --image=zhsunregistry.azurecr.io/hello-acr:latest --allow-missing-images -n e2e-test-kubelet-authentication-providers-mpvxd'
 W0507 14:15:14.838662   61979 newapp.go:523] Could not find an image stream match for "zhsunregistry.azurecr.io/hello-acr:latest". Make sure that a container image with that tag is available on the node for the deployment to succeed.
 --> Found container image 7af3297 (6 years old) from zhsunregistry.azurecr.io for "zhsunregistry.azurecr.io/hello-acr:latest"


 --> Creating resources ...
     deployment.apps "hello-acr" created
     service "hello-acr" created
 --> Success
     Application is not exposed. You can expose services to the outside world by executing one or more of the commands below:
      'oc expose service/hello-acr' 
     Run 'oc status' to view your app.
   STEP: Wait the pod to be running @ 05/07/24 14:15:15.513
 May  7 14:15:17.205: INFO: Deleted {user.openshift.io/v1, Resource=users  e2e-test-kubelet-authentication-providers-mpvxd-user}, err: <nil>
 May  7 14:15:17.472: INFO: Deleted {oauth.openshift.io/v1, Resource=oauthclients  e2e-client-e2e-test-kubelet-authentication-providers-mpvxd}, err: <nil>
 May  7 14:15:17.742: INFO: Deleted {oauth.openshift.io/v1, Resource=oauthaccesstokens  sha256~gjXih-PLLeHKO3vfK2jiSXQUMXm4Eh4zEtIfbNv0iMM}, err: <nil>
   STEP: Destroying namespace "e2e-test-kubelet-authentication-providers-mpvxd" for this suite. @ 05/07/24 14:15:17.743
 • [26.603 seconds]
 ------------------------------

 Ran 1 of 1 Specs in 26.603 seconds
 SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot avatar May 07 '24 06:05 openshift-ci-robot

I can't see the new tests in the JUnit output, are they showing for you?

JoelSpeed avatar May 07 '24 09:05 JoelSpeed

I can't see the new tests in the JUnit output, are they showing for you?

Seems need to run make update so that test/extended/util/annotate/generated/zz_generated.annotations.go can be updated, now I can see

$ ./openshift-tests run --dry-run "openshift/conformance/serial" | grep -E KubeletAuthenticationProviders    
INFO[0000] Decoding provider                             clusterState="<nil>" discover=true dryRun=true func=DecodeProvider providerType=
"[sig-cluster-lifecycle][Feature:KubeletAuthenticationProviders][Serial] KubeletAuthenticationProviders should should be able to pull images from ACR [Suite:openshift/conformance/serial]"
"[sig-cluster-lifecycle][Feature:KubeletAuthenticationProviders][Serial] KubeletAuthenticationProviders should should be able to pull images from ECR [Suite:openshift/conformance/serial]"
"[sig-cluster-lifecycle][Feature:KubeletAuthenticationProviders][Serial] KubeletAuthenticationProviders should should be able to pull images from GCR [Suite:openshift/conformance/serial]"

sunzhaohua2 avatar May 07 '24 11:05 sunzhaohua2

Great, can see now that the test is failing, so lets try to get the images uploaded so we can get this to pass, thanks!

JoelSpeed avatar May 07 '24 14:05 JoelSpeed

Great, can see now that the test is failing, so lets try to get the images uploaded so we can get this to pass, thanks!

asked here https://redhat-internal.slack.com/archives/CBN38N3MW/p1715133269211149

sunzhaohua2 avatar May 08 '24 01:05 sunzhaohua2

Job Failure Risk Analysis for sha: 52c34a9c29ac7ba9e76cf0e771b0557cf4414b10

Job Name Failure Risk
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-upgrade High
[bz-Node Tuning Operator] clusteroperator/node-tuning should not change condition/Available
This test has passed 99.67% of 3360 runs on release 4.17 [Overall] in the last week.
---
[sig-arch] events should not repeat pathologically for ns/openshift-kube-apiserver-operator
This test has passed 99.67% of 3358 runs on release 4.17 [Overall] in the last week.

openshift-trt-bot avatar May 20 '24 17:05 openshift-trt-bot

@sunzhaohua2: The specified target(s) for /test were not found. The following commands are available to trigger required jobs:

  • /test e2e-aws-jenkins
  • /test e2e-aws-ovn-edge-zones
  • /test e2e-aws-ovn-fips
  • /test e2e-aws-ovn-image-registry
  • /test e2e-aws-ovn-serial
  • /test e2e-gcp-ovn
  • /test e2e-gcp-ovn-builds
  • /test e2e-gcp-ovn-image-ecosystem
  • /test e2e-gcp-ovn-upgrade
  • /test e2e-metal-ipi-ovn-ipv6
  • /test images
  • /test lint
  • /test unit
  • /test verify
  • /test verify-deps

The following commands are available to trigger optional jobs:

  • /test 4.12-upgrade-from-stable-4.11-e2e-aws-ovn-upgrade-rollback
  • /test e2e-agnostic-ovn-cmd
  • /test e2e-aws
  • /test e2e-aws-csi
  • /test e2e-aws-disruptive
  • /test e2e-aws-etcd-recovery
  • /test e2e-aws-multitenant
  • /test e2e-aws-ovn
  • /test e2e-aws-ovn-cgroupsv2
  • /test e2e-aws-ovn-etcd-scaling
  • /test e2e-aws-ovn-kubevirt
  • /test e2e-aws-ovn-single-node
  • /test e2e-aws-ovn-single-node-serial
  • /test e2e-aws-ovn-single-node-upgrade
  • /test e2e-aws-ovn-upgrade
  • /test e2e-aws-ovn-upi
  • /test e2e-aws-proxy
  • /test e2e-azure
  • /test e2e-azure-ovn-etcd-scaling
  • /test e2e-azure-ovn-upgrade
  • /test e2e-baremetalds-kubevirt
  • /test e2e-gcp-csi
  • /test e2e-gcp-disruptive
  • /test e2e-gcp-fips-serial
  • /test e2e-gcp-ovn-etcd-scaling
  • /test e2e-gcp-ovn-rt-upgrade
  • /test e2e-gcp-ovn-techpreview
  • /test e2e-gcp-ovn-techpreview-serial
  • /test e2e-metal-ipi-ovn-dualstack
  • /test e2e-metal-ipi-ovn-dualstack-local-gateway
  • /test e2e-metal-ipi-sdn
  • /test e2e-metal-ipi-serial
  • /test e2e-metal-ipi-serial-ovn-ipv6
  • /test e2e-metal-ipi-virtualmedia
  • /test e2e-openstack-ovn
  • /test e2e-openstack-serial
  • /test e2e-vsphere
  • /test e2e-vsphere-ovn-dualstack-primaryv6
  • /test e2e-vsphere-ovn-etcd-scaling
  • /test okd-e2e-gcp
  • /test okd-scos-images

Use /test all to run the following jobs that were automatically triggered:

  • pull-ci-openshift-origin-master-e2e-agnostic-ovn-cmd
  • pull-ci-openshift-origin-master-e2e-aws-csi
  • pull-ci-openshift-origin-master-e2e-aws-ovn-cgroupsv2
  • pull-ci-openshift-origin-master-e2e-aws-ovn-edge-zones
  • pull-ci-openshift-origin-master-e2e-aws-ovn-fips
  • pull-ci-openshift-origin-master-e2e-aws-ovn-serial
  • pull-ci-openshift-origin-master-e2e-aws-ovn-single-node
  • pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-serial
  • pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-upgrade
  • pull-ci-openshift-origin-master-e2e-aws-ovn-upgrade
  • pull-ci-openshift-origin-master-e2e-gcp-csi
  • pull-ci-openshift-origin-master-e2e-gcp-ovn
  • pull-ci-openshift-origin-master-e2e-gcp-ovn-builds
  • pull-ci-openshift-origin-master-e2e-gcp-ovn-rt-upgrade
  • pull-ci-openshift-origin-master-e2e-gcp-ovn-upgrade
  • pull-ci-openshift-origin-master-e2e-metal-ipi-ovn-ipv6
  • pull-ci-openshift-origin-master-e2e-metal-ipi-sdn
  • pull-ci-openshift-origin-master-e2e-openstack-ovn
  • pull-ci-openshift-origin-master-images
  • pull-ci-openshift-origin-master-lint
  • pull-ci-openshift-origin-master-unit
  • pull-ci-openshift-origin-master-verify
  • pull-ci-openshift-origin-master-verify-deps

In response to this:

/test e2e-azure-ovn-serial

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci[bot] avatar May 20 '24 23:05 openshift-ci[bot]

Job Failure Risk Analysis for sha: ea95b6292a9e33c2bf0a70c31f2d7b1eb90cc804

Job Name Failure Risk
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-upgrade High
[bz-Node Tuning Operator] clusteroperator/node-tuning should not change condition/Available
This test has passed 99.60% of 3458 runs on release 4.17 [Overall] in the last week.
---
[sig-arch] events should not repeat pathologically for ns/openshift-kube-apiserver-operator
This test has passed 99.57% of 3454 runs on release 4.17 [Overall] in the last week.
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-serial IncompleteTests
Tests for this run (98) are below the historical average (770): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node IncompleteTests
Tests for this run (98) are below the historical average (1617): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

openshift-trt-bot avatar May 21 '24 06:05 openshift-trt-bot

@sunzhaohua2: The specified target(s) for /test were not found. The following commands are available to trigger required jobs:

  • /test e2e-aws-jenkins
  • /test e2e-aws-ovn-edge-zones
  • /test e2e-aws-ovn-fips
  • /test e2e-aws-ovn-image-registry
  • /test e2e-aws-ovn-serial
  • /test e2e-gcp-ovn
  • /test e2e-gcp-ovn-builds
  • /test e2e-gcp-ovn-image-ecosystem
  • /test e2e-gcp-ovn-upgrade
  • /test e2e-metal-ipi-ovn-ipv6
  • /test images
  • /test lint
  • /test unit
  • /test verify
  • /test verify-deps

The following commands are available to trigger optional jobs:

  • /test 4.12-upgrade-from-stable-4.11-e2e-aws-ovn-upgrade-rollback
  • /test e2e-agnostic-ovn-cmd
  • /test e2e-aws
  • /test e2e-aws-csi
  • /test e2e-aws-disruptive
  • /test e2e-aws-etcd-recovery
  • /test e2e-aws-multitenant
  • /test e2e-aws-ovn
  • /test e2e-aws-ovn-cgroupsv2
  • /test e2e-aws-ovn-etcd-scaling
  • /test e2e-aws-ovn-kubevirt
  • /test e2e-aws-ovn-single-node
  • /test e2e-aws-ovn-single-node-serial
  • /test e2e-aws-ovn-single-node-upgrade
  • /test e2e-aws-ovn-upgrade
  • /test e2e-aws-ovn-upi
  • /test e2e-aws-proxy
  • /test e2e-azure
  • /test e2e-azure-ovn-etcd-scaling
  • /test e2e-azure-ovn-upgrade
  • /test e2e-baremetalds-kubevirt
  • /test e2e-gcp-csi
  • /test e2e-gcp-disruptive
  • /test e2e-gcp-fips-serial
  • /test e2e-gcp-ovn-etcd-scaling
  • /test e2e-gcp-ovn-rt-upgrade
  • /test e2e-gcp-ovn-techpreview
  • /test e2e-gcp-ovn-techpreview-serial
  • /test e2e-metal-ipi-ovn-dualstack
  • /test e2e-metal-ipi-ovn-dualstack-local-gateway
  • /test e2e-metal-ipi-sdn
  • /test e2e-metal-ipi-serial
  • /test e2e-metal-ipi-serial-ovn-ipv6
  • /test e2e-metal-ipi-virtualmedia
  • /test e2e-openstack-ovn
  • /test e2e-openstack-serial
  • /test e2e-vsphere
  • /test e2e-vsphere-ovn-dualstack-primaryv6
  • /test e2e-vsphere-ovn-etcd-scaling
  • /test okd-e2e-gcp
  • /test okd-scos-images

Use /test all to run the following jobs that were automatically triggered:

  • pull-ci-openshift-origin-master-e2e-agnostic-ovn-cmd
  • pull-ci-openshift-origin-master-e2e-aws-csi
  • pull-ci-openshift-origin-master-e2e-aws-ovn-cgroupsv2
  • pull-ci-openshift-origin-master-e2e-aws-ovn-edge-zones
  • pull-ci-openshift-origin-master-e2e-aws-ovn-fips
  • pull-ci-openshift-origin-master-e2e-aws-ovn-serial
  • pull-ci-openshift-origin-master-e2e-aws-ovn-single-node
  • pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-serial
  • pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-upgrade
  • pull-ci-openshift-origin-master-e2e-aws-ovn-upgrade
  • pull-ci-openshift-origin-master-e2e-gcp-csi
  • pull-ci-openshift-origin-master-e2e-gcp-ovn
  • pull-ci-openshift-origin-master-e2e-gcp-ovn-builds
  • pull-ci-openshift-origin-master-e2e-gcp-ovn-rt-upgrade
  • pull-ci-openshift-origin-master-e2e-gcp-ovn-upgrade
  • pull-ci-openshift-origin-master-e2e-metal-ipi-ovn-ipv6
  • pull-ci-openshift-origin-master-e2e-metal-ipi-sdn
  • pull-ci-openshift-origin-master-e2e-openstack-ovn
  • pull-ci-openshift-origin-master-images
  • pull-ci-openshift-origin-master-lint
  • pull-ci-openshift-origin-master-unit
  • pull-ci-openshift-origin-master-verify
  • pull-ci-openshift-origin-master-verify-deps

In response to this:

/test e2e-azure-fips-serial

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci[bot] avatar May 21 '24 07:05 openshift-ci[bot]

/payload-job periodic-ci-openshift-release-master-ci-4.17-e2e-azure-ovn-serial

JoelSpeed avatar May 21 '24 08:05 JoelSpeed

@JoelSpeed: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.17-e2e-azure-ovn-serial

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/06908cd0-174a-11ef-9adf-a5c46a74c547-0

openshift-ci[bot] avatar May 21 '24 08:05 openshift-ci[bot]

Job Failure Risk Analysis for sha: 8f4fa63bb915a345a9df72a921a97ad116607ce0

Job Name Failure Risk
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-upgrade High
[bz-Node Tuning Operator] clusteroperator/node-tuning should not change condition/Available
This test has passed 99.59% of 3384 runs on release 4.17 [Overall] in the last week.
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node IncompleteTests
Tests for this run (98) are below the historical average (1598): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

openshift-trt-bot avatar May 21 '24 11:05 openshift-trt-bot

/payload-job periodic-ci-openshift-release-master-nightly-4.17-e2e-gcp-ovn-serial

sunzhaohua2 avatar May 22 '24 02:05 sunzhaohua2

@sunzhaohua2: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.17-e2e-gcp-ovn-serial

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/2b85aa90-17e3-11ef-8a5a-54c02cf856df-0

openshift-ci[bot] avatar May 22 '24 02:05 openshift-ci[bot]

Job Failure Risk Analysis for sha: 7e362f55a4a57582b8947a8c34424859e556c7cd

Job Name Failure Risk
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-upgrade High
[bz-Node Tuning Operator] clusteroperator/node-tuning should not change condition/Available
This test has passed 99.63% of 3780 runs on release 4.17 [Overall] in the last week.
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node IncompleteTests
Tests for this run (98) are below the historical average (1551): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

openshift-trt-bot avatar May 22 '24 06:05 openshift-trt-bot

/payload-job periodic-ci-openshift-release-master-nightly-4.17-e2e-aws-ovn-serial

sunzhaohua2 avatar May 22 '24 06:05 sunzhaohua2

@sunzhaohua2: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.17-e2e-aws-ovn-serial

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/9e14bdf0-1804-11ef-8b77-f885f8f851e4-0

openshift-ci[bot] avatar May 22 '24 06:05 openshift-ci[bot]

Job Failure Risk Analysis for sha: 4c2c27394ecbbf4125dd456d070d28b3171c2e06

Job Name Failure Risk
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-upgrade High
[bz-Node Tuning Operator] clusteroperator/node-tuning should not change condition/Available
This test has passed 99.62% of 3731 runs on release 4.17 [Overall] in the last week.
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-serial IncompleteTests
Tests for this run (98) are below the historical average (764): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node IncompleteTests
Tests for this run (98) are below the historical average (1493): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

openshift-trt-bot avatar May 22 '24 16:05 openshift-trt-bot

/test e2e-aws-ovn-serial

jupierce avatar Jul 09 '24 15:07 jupierce

@sunzhaohua2 Do you remember where we got to with this?

JoelSpeed avatar Jul 10 '24 07:07 JoelSpeed

@sunzhaohua2 Do you remember where we got to with this?

  • Only azure passed.
  • AWS met permission issue, ticket https://issues.redhat.com/browse/DPTP-4043 AccessDeniedException: User: arn:aws:iam::892173657978:user/origin-ci-robot-provision is not authorized to perform: ecr:CreateRepository on resource: arn:aws:ecr:us-west-2:892173657978:repository/ecr-registry because no identity-based policy allows the ecr:CreateRepository action
  • GCP met permission issue, I will check again if I can resolve it. error: unable to push quay.io/openshifttest/pause: failed to upload blob sha256:cc40f7bda7213c7f8e7ecd07faccfa60c0adc51b007b70786f51b408a5d7d8e4: Post "https://gcr.io/v2/XXXXXXXXXXXXXXXXXXXXXXXX/gcr-registry/blobs/uploads/": unauthorized: Not Authorized.

sunzhaohua2 avatar Jul 10 '24 13:07 sunzhaohua2

Job Failure Risk Analysis for sha: 1a751ef4b08b0ac68238d3b5bb36625052d1f24b

Job Name Failure Risk
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-upgrade High
[sig-cluster-lifecycle] pathological event should not see excessive Back-off restarting failed containers for ns/openshift-marketplace
This test has passed 99.68% of 5077 runs on release 4.17 [Overall] in the last week.

openshift-trt-bot avatar Jul 10 '24 17:07 openshift-trt-bot

I added ECR permissions to the core product engineering CI accounts, so hopefully you will no longer see the CreateRepository error: https://issues.redhat.com/browse/DPTP-4043?focusedId=25102958&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-25102958 .

jupierce avatar Jul 10 '24 22:07 jupierce

I added ECR permissions to the core product engineering CI accounts, so hopefully you will no longer see the CreateRepository error: https://issues.redhat.com/browse/DPTP-4043?focusedId=25102958&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-25102958 .

Thank you, it passed in job 1811037317838671872

sunzhaohua2 avatar Jul 11 '24 02:07 sunzhaohua2

/payload-job periodic-ci-openshift-release-master-ci-4.17-e2e-azure-ovn-serial

sunzhaohua2 avatar Jul 11 '24 15:07 sunzhaohua2

/payload-job periodic-ci-openshift-release-master-nightly-4.17-e2e-gcp-ovn-serial

sunzhaohua2 avatar Jul 11 '24 15:07 sunzhaohua2

@sunzhaohua2: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.17-e2e-gcp-ovn-serial

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/86cb2800-3f96-11ef-9dd5-db60e60405c1-0

openshift-ci[bot] avatar Jul 11 '24 15:07 openshift-ci[bot]

@sunzhaohua2: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.17-e2e-azure-ovn-serial

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/7eac9aa0-3f96-11ef-8336-4d7d99785651-0

openshift-ci[bot] avatar Jul 11 '24 15:07 openshift-ci[bot]

Job Failure Risk Analysis for sha: 1413829c2599b92699335507d8eab2a6939b4238

Job Name Failure Risk
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node High
Undiagnosed panic detected in pod
This test has passed 100.00% of 37 runs on jobs ['periodic-ci-openshift-release-master-nightly-4.17-e2e-aws-ovn-single-node'] in the last 14 days.

Open Bugs
openshift-controller-manager pod panic due to type assertion

openshift-trt-bot avatar Jul 12 '24 04:07 openshift-trt-bot

The tests passed on aws gcp azure , failed on aws sno , raised a bug https://issues.redhat.com/browse/OCPBUGS-36906

sunzhaohua2 avatar Jul 12 '24 05:07 sunzhaohua2