Add e2e test (33431) for ComplianceCheckResult label queries

[taimurhafeez]

Verify that ComplianceCheckResult objects can be queried using labels:

• compliance.openshift.io/check-severity
• compliance.openshift.io/check-status
• compliance.openshift.io/scan-name
• compliance.openshift.io/suite

Changes

• Add TestComplianceCheckResultLabels test in tests/e2e/serial/main_test.go
• Add AssertCheckResultByLabel helper in tests/e2e/framework/common.go

-to implement this PR

1. The following line needs to be commented in tests/e2e/framework/main_entry.go or there will be error error creating Machine Config Pool e2e-invalid line to be commented --> return fmt.Errorf("failed to create Machine Config Pool %s: %w", "e2e-invalid", err)
2. Run the test in make e2e-serial E2E_GO_TEST_FLAGS="-v -run TestComplianceCheckResultLabels"

[openshift-ci[bot]]

Hi @taimurhafeez. Thanks for your PR.

I'm waiting for a github.com member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[xiaojiey]

Hi @taimurhafeez, I don't think it is meaningful to create a dedicate serial test case to check the labels for the ccr. If you really want to cover the test point, I think it is better to adding label queries to 1 existing parallel tests than creating a dedicated serial test

[openshift-ci[bot]]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: taimurhafeez Once this PR has been reviewed and has the lgtm label, please assign vincent056 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[Anna-Koudelkova]

The test case is PASSing succesfully on OCP 4.20 cluster. After running it, one of the worker nodes on my cluster has gone into degraded state though:

$ oc get mcp
NAME     CONFIG                                             UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT   AGE
master   rendered-master-2b8ffc374493b63e18f96d2aa594f092   True      False      False      3              3                   3                     0                      102m
worker   rendered-worker-033af400c2a339ce01799d80ce78313c   False     True       True       3              2                   2                     1                      102m

I am not really sure if this is just a coincidence or what is a rootcase yet, so far I just want to post it here so we have a trace that this has happened - in case it happens to someone else as well in the future

[openshift-ci-robot]

@taimurhafeez: This pull request references CMP-3800 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.21.0" version, but no target version was set.

In response to this:

Verify that ComplianceCheckResult objects can be queried using labels:

• compliance.openshift.io/check-severity
• compliance.openshift.io/check-status
• compliance.openshift.io/scan-name
• compliance.openshift.io/suite

Changes

• Updated TestScanProducesRemediations to TestScanProducesRemediationsAndLabels in tests/e2e/parallel/main_test.go
• Add AssertCheckResultByLabel helper in tests/e2e/framework/common.go

-to implement this PR

1. The following line needs to be commented in tests/e2e/framework/main_entry.go or there will be error error creating Machine Config Pool e2e-invalid. line to be commented --> return fmt.Errorf("failed to create Machine Config Pool %s: %w", "e2e-invalid", err)
2. Run the test in make e2e-serial E2E_GO_TEST_FLAGS="-v -run TestScanProducesRemediationsAndLabels"

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[taimurhafeez]

Hi @taimurhafeez, I don't think it is meaningful to create a dedicate serial test case to check the labels for the ccr. If you really want to cover the test point, I think it is better to adding label queries to 1 existing parallel tests than creating a dedicated serial test I added to the already TestScanProducesRemediations test but renamed it to TestScanProducesRemediationsAndLabels

[taimurhafeez]

The test case is PASSing succesfully on OCP 4.20 cluster. After running it, one of the worker nodes on my cluster has gone into degraded state though:

$ oc get mcp
NAME     CONFIG                                             UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT   AGE
master   rendered-master-2b8ffc374493b63e18f96d2aa594f092   True      False      False      3              3                   3                     0                      102m
worker   rendered-worker-033af400c2a339ce01799d80ce78313c   False     True       True       3              2                   2                     1                      102m

I am not really sure if this is just a coincidence or what is a rootcase yet, so far I just want to post it here so we have a trace that this has happened - in case it happens to someone else as well in the future @Anna-Koudelkova It is the same with me on 4.20. I dug further, and a possible reason I found is maybe it uses ocp4-moderate, which affects the cluster state, but the test itself is in parallel?

[rhmdnd]

Potential transient issue:

 === NAME  TestScanSettingBindingNoStorage
    main_test.go:4184: Expected to find PVC associated with the scan.
2025/12/02 14:16:24 waiting until suite test-custom-rule-with-multiple-inputs-ssb reaches target status 'DONE'. Current status: RUNNING
--- FAIL: TestScanSettingBindingNoStorage (105.17s) 

[taimurhafeez]

Potential transient issue:

 === NAME  TestScanSettingBindingNoStorage
    main_test.go:4184: Expected to find PVC associated with the scan.
2025/12/02 14:16:24 waiting until suite test-custom-rule-with-multiple-inputs-ssb reaches target status 'DONE'. Current status: RUNNING
--- FAIL: TestScanSettingBindingNoStorage (105.17s) 

possibly because I ran recently, and it went through

=== RUN   TestScanProducesRemediationsAndLabels
=== PAUSE TestScanProducesRemediationsAndLabels
=== CONT  TestScanProducesRemediationsAndLabels
2025/12/03 13:33:44 waiting until suite test-scan-produces-remediations-and-labels reaches target status 'DONE'. Current status: RUNNING
2025/12/03 13:33:49 waiting until suite test-scan-produces-remediations-and-labels reaches target status 'DONE'. Current status: RUNNING
2025/12/03 13:33:54 waiting until suite test-scan-produces-remediations-and-labels reaches target status 'DONE'. Current status: RUNNING
2025/12/03 13:33:59 waiting until suite test-scan-produces-remediations-and-labels reaches target status 'DONE'. Current status: RUNNING
2025/12/03 13:34:04 waiting until suite test-scan-produces-remediations-and-labels reaches target status 'DONE'. Current status: RUNNING
2025/12/03 13:34:09 waiting until suite test-scan-produces-remediations-and-labels reaches target status 'DONE'. Current status: RUNNING
2025/12/03 13:34:14 waiting until suite test-scan-produces-remediations-and-labels reaches target status 'DONE'. Current status: AGGREGATING
2025/12/03 13:34:19 waiting until suite test-scan-produces-remediations-and-labels reaches target status 'DONE'. Current status: AGGREGATING
2025/12/03 13:34:24 waiting until suite test-scan-produces-remediations-and-labels reaches target status 'DONE'. Current status: AGGREGATING
2025/12/03 13:34:29 waiting until suite test-scan-produces-remediations-and-labels reaches target status 'DONE'. Current status: AGGREGATING
2025/12/03 13:34:39 ComplianceScan ready (DONE)
2025/12/03 13:34:39 All scans in ComplianceSuite have finished (test-scan-produces-remediations-and-labels)
osdk-e2e-b9e29c29-69f4-4190-b7b2-b6c36fc2663e map[compliance.openshift.io/suite:test-scan-produces-remediations-and-labels]
--- PASS: TestScanProducesRemediationsAndLabels (62.58s)
PASS

[xiaojiey]

/retest

[openshift-ci[bot]]

@taimurhafeez: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-rosa	0a5878d7a9cb5b0a2aa307a351d724d5975f55d6	link	true	/test e2e-rosa
ci/prow/e2e-aws-serial	0a5878d7a9cb5b0a2aa307a351d724d5975f55d6	link	true	/test e2e-aws-serial

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.