kubernetes icon indicating copy to clipboard operation
kubernetes copied to clipboard

test: Fix image credential pulls test node scheduling

Open saschagrunert opened this issue 1 month ago • 11 comments

What type of PR is this?

/kind failing-test

What this PR does / why we need it:

Fixes a node scheduling bug in the image credential pulls test that was causing test timeouts in the serial CRI-O node e2e suite.

The registry and test pods were being scheduled on different nodes. When the test pod tried to pull images from localhost:5000, it couldn't reach the registry running on a different node, causing test failures.

The fix ensures both the registry and test pods run on the same node by using the registry node for scheduling the test pod. In node e2e tests, this is the single test node, so proper affinity is established.

Which issue(s) this PR is related to:

Fixes #135375

Special notes for your reviewer:

  • Related PR: #135244 (identical node scheduling fix for a different test)
  • Test-infra fixes: kubernetes/test-infra#35946 (CRI-O insecure registry config), kubernetes/test-infra#35969 (CRI-O upgrade including cri-o/cri-o#9615)

Does this PR introduce a user-facing change?

None

Additional documentation:

None

saschagrunert avatar Nov 20 '25 13:11 saschagrunert

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: saschagrunert Once this PR has been reviewed and has the lgtm label, please assign derekwaynecarr for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot avatar Nov 27 '25 08:11 k8s-ci-robot

/test pull-kubernetes-node-kubelet-serial-crio

saschagrunert avatar Nov 27 '25 08:11 saschagrunert

The serial tests still fail, but most of the infra issues are now gone. We have to follow-up on the existing failures/flakes.

PTAL @kubernetes/sig-node-cri-o-test-maintainers @kubernetes/sig-node-test-failures @kubernetes/sig-node-pr-reviews

saschagrunert avatar Nov 27 '25 10:11 saschagrunert

So this seems important for 1.35?

I see a minor product code change in addition to test changes.

And we see that CRI-O tests are failing due to this.

Should we merge this into 1.35 or carry a patch?

/assign @stlaz @aramase

kannon92 avatar Dec 10 '25 16:12 kannon92

/priority important-soon

kannon92 avatar Dec 10 '25 16:12 kannon92

/triage accepted

kannon92 avatar Dec 10 '25 16:12 kannon92

/lgtm

we may want to wait for 1.36 branch to open then to backport it at this stage

haircommander avatar Dec 10 '25 18:12 haircommander

LGTM label has been added.

Git tree hash: 1163143c26472e35d9b552bca7e5f3216863fce5

k8s-ci-robot avatar Dec 10 '25 18:12 k8s-ci-robot

/skip

saschagrunert avatar Dec 10 '25 19:12 saschagrunert

@saschagrunert: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-kubernetes-node-kubelet-serial-crio c4a4d00ebd19e0b804cc2abc190657a393f14641 link false /test pull-kubernetes-node-kubelet-serial-crio
pull-kubernetes-e2e-gce cb011623c845c88c42c7c04242e3bdfb0cd1ffa2 link true /test pull-kubernetes-e2e-gce
pull-kubernetes-node-e2e-containerd cb011623c845c88c42c7c04242e3bdfb0cd1ffa2 link true /test pull-kubernetes-node-e2e-containerd

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

k8s-ci-robot avatar Dec 11 '25 15:12 k8s-ci-robot

LGTM label has been added.

Git tree hash: a17f7aef872c63b699f952e4dfd4d966854532cf

k8s-ci-robot avatar Dec 11 '25 15:12 k8s-ci-robot