Add ATEX testing to the upstream CI workflows

[ggbecker]

Description:

• Add ATEX testing to the upstream CI workflows
• It posts the resulting tests link as a comment to the pull request when it finishes.

Rationale:

• This aims to replace existing testing farm individual checks to a centralized ATEX execution that can run all CI upstream tests for PRs and manage them accordingly.
• The PR as of now, runs only a single STIG hardening test on Centos Stream 8/9/10 to be served as a proof of concept. We should eventually extend to include more tests similarly as the current upstream CI.

[ggbecker]

From: https://github.com/ComplianceAsCode/content/actions/runs/19858710726/job/56915038821?pr=14203

   File "/usr/local/lib/python3.13/site-packages/atex/provisioner/testingfarm/api.py", line 507, in reserve
    if self.api.whoami()["token"]["ranch"] == "public":
       ~~~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.13/site-packages/atex/provisioner/testingfarm/api.py", line 122, in whoami
    raise ValueError("whoami() requires an auth token")
ValueError: whoami() requires an auth token

I'm afraid the token will only be allowed to be used when we merge the pull request

https://github.com/ComplianceAsCode/content/pull/14203/files#diff-9581118f6672453f95900179c8eccc554c1b111682dab06bd16317909fdaf295R109

The same code with the same token is working fine on my fork: https://github.com/ggbecker/content/pull/41

[ggbecker]

I've addressed all the feedback provided and split the jobs into two workflows, one with the workflow_run as suggested by @Mab879 . Let's see how it behaves.

[comps]

How would I make a change in the workflow (later down the road) while being able to test the change in a PR, and not hit any of the secrets limitations?

For example, if we switch to pip install atex==0.20 or so, and I want to upgrade to pip install atex==1.0, which brings an incompatible API change, how can I change the workflow code (to adjust for new API) while being able to test the change?

Will it Just Work?

---

Also nitpicks:

• Contest is technically a "test suite", not a "framework" (though, arguably, Contest contains a framework inside lib/)
• This PR is about running Contest in upstream PRs, not about running "Testing Farm", ... TF is one of the backends we can use to run Contest, but not a type of test / suite, so the bot comment is slightly confusing

[ggbecker]

How would I make a change in the workflow (later down the road) while being able to test the change in a PR, and not hit any of the secrets limitations?

For example, if we switch to pip install atex==0.20 or so, and I want to upgrade to pip install atex==1.0, which brings an incompatible API change, how can I change the workflow code (to adjust for new API) while being able to test the change?

Will it Just Work?

I believe it will not. Maybe if you push to a branch belonging to ComplianceAsCode the situation changes, at least that's what worked here https://github.com/ComplianceAsCode/content/pull/14209.

But then I don't know about the workflow_run type that may behave a bit differently. I'm going to investigate this better to see if I can find the relevant information.

[ggbecker]

/packit retest-failed

[Mab879]

I believe it will not. Maybe if you push to a branch belonging to ComplianceAsCode the situation changes, at least that's what worked here #14209.

Sadly, this is a feature not bug. So that we don't expose our secrets.

[ggbecker]

@Mab879 removing the branches seems to be the solution for the workflow to be triggered: c51e5be

This might also be the solution for the compare-ds job https://github.com/ComplianceAsCode/content/blob/828571f288561fa99902f5bc6f2c2681d3dde908/.github/workflows/compare-ds.yaml#L7

On my fork it worked: https://github.com/ggbecker/content/pull/44

[Mab879]

@Mab879 removing the branches seems to be the solution for the workflow to be triggered: c51e5be

This might also be the solution for the compare-ds job

https://github.com/ComplianceAsCode/content/blob/828571f288561fa99902f5bc6f2c2681d3dde908/.github/workflows/compare-ds.yaml#L7

On my fork it worked: ggbecker#44

Makes sense. I'm fine with that change.

[ggbecker]

I believe this is now ready to be merged. Of course some improvements can be done, but the main functionality is there: see here https://github.com/ggbecker/content/pull/45

[openshift-ci[bot]]

@ggbecker: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-aws-openshift-node-compliance	022cebbfb46d926f5051837d03c3b4d3f797c6f8	link	true	/test e2e-aws-openshift-node-compliance

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.