Library files output that no policies were found

[jpreese]

In v0.20.0 and v0.21.0 of Conftest, the output for running a test that included library files did not render to the console that no policies were found:

❯ ./conftest test -p examples/policies ok.yaml --all-namespaces
FAIL - deployment.yaml - P1001: Deployment/hello-kubernetes/hello-kubernetes: Does not drop all capabilities
FAIL - deployment.yaml - P2002: Deployment/hello-kubernetes/hello-kubernetes: Container resource constraints must be specified
FAIL - deployment.yaml - P1008: Deployment/hello-kubernetes: Pod allows running as root
FAIL - deployment.yaml - P1009: Deployment/hello-kubernetes: Does not require droping all capabilities

However, in v0.22.0 it does:

? - deployment.yaml - lib.security - no policies found
? - deployment.yaml - lib.core - no policies found
? - deployment.yaml - lib.pods - no policies found
FAIL - deployment.yaml - container_deny_added_caps - P1001: Deployment/hello-kubernetes/hello-kubernetes: Does not drop all capabilities
? - deployment.yaml - lib.psps - no policies found
FAIL - deployment.yaml - pod_deny_without_runasnonroot - P1008: Deployment/hello-kubernetes: Pod allows running as root
FAIL - deployment.yaml - psp_deny_added_caps - P1009: Deployment/hello-kubernetes: Does not require droping all capabilities
? - deployment.yaml - lib.measurements - no policies found
FAIL - deployment.yaml - container_deny_without_resource_constraints - P2002: Deployment/hello-kubernetes/hello-kubernetes: Container resource constraints must be specified
? - deployment.yaml - lib.rbac - no policies found

0.21: https://github.com/open-policy-agent/conftest/blob/125160deacb9c02ce3c098bdf1f3ce7df216026a/internal/runner/test.go#L92

0.22: https://github.com/open-policy-agent/conftest/blob/07a60e75a625f1e5d21591c2cf60bb68c38e7c62/internal/runner/test.go#L70

@jalseth @06kellyjac

[EmilMunksoe]

Any update on this. This is quite annoying to work with. Let me know if I can be of any assistance. I will gladly help contribute if someone can point me in the right direction.

[jpreese]

Is it just the verbose output? The end result should still be valid.

I may be able to look at this again soon if it's causing that much pain, but it would most likely take some effort to resolve and keep the current behavior of letting users know when policies haven't been found.

[EmilMunksoe]

Yeah the end result is still valid, it is just verbose output that is not being filtered out.

[Banders2]

@jpreese We just ran into this "issue", did you ever get around to taking a look at it?