[Feature] Run test conditionally

[karentu2]

Problem Statement

For example, I want to run a chainsaw test if a Kyverno policy is in Enforce mode, but if the Kyverno policy is in Audit mode, then the chainsaw test is not run.

Solution Description

One way I can think of to implement this is to add some precondition check field under spec, in the example below I named it precheck. So if the precheck fails, then the test is skipped.

apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Test
metadata:
  name: example
spec:
  precheck:
  - try:
    - assert:
        file: path/to/kyverno-policy-enforce-mode.yaml # where this file indicates that a kyverno policy is in Enforce mode
  steps:
  - try:
    - apply:
        file: path/to/apply.yaml
    - assert:
        file: path/to/assert.yaml
    catch: []
    finally: []

Alternatives

It might be better to have the precheck field available on both the spec level so that the entire test can be skipped, and available on the spec.steps level so that specific steps can be skipped.

Additional Context

No response

Slack discussion

https://kubernetes.slack.com/archives/C067LUFL43U/p1712276658755649

Research

• [X] I have searched other issues in this repository and mine is not recorded.