conftest

conftest copied to clipboard

I would like to be able to emit a human readable message for Exceptions so it's easy to see why something has been excluded. Currently it's not very obvious.

What version of Conftest are you using? The master branch already solves this issue by showing the exception triggered:

EXCP - examples/exceptions/deployments.yaml - data.main.exception[_][_] == "run_as_root"

Maybe this is already enough to cover your use case, it would at least indicate from what rule the configuration object was excepted. Or do you have other use cases that require a bit more control over the output of the exceptions?

@Blokje5 I am using 0.21.0.

I have just tried master as suggested. Unfortunately this doesn't help me enough as it only works if you're testing files, but I'm using kustomize to build the manifests, for example:

$ kustomize build thing | conftest test -
EXCP - data.main.exception[_][_] == "missing_memory_limits"
EXCP - data.main.exception[_][_] == "missing_cpu_limits"

I would like to be able to use sprintf to customize the msg with string values.

A lightweight pseudo example of what I'd like to be able to do:

deny_missing_memory_limits[msg] {
	kubernetes.containers[container]
	not container.resources.limits.memory
	msg = sprintf("%s in the %s %s does not have a memory limit set", [container.name, kubernetes.kind, kubernetes.name])
}

deny_missing_cpu_limits[msg] {
	kubernetes.containers[container]
	not container.resources.limits.cpu
	msg = sprintf("%s in the %s %s does not have a CPU limit set", [container.name, kubernetes.kind, kubernetes.name])
}

# exception for my-special-deployment limits due to performance issues
exception[rules, msg] {
  input.kind = "Deployment"
  input.metadata.name = "my-special-deployment"
  reason = "some performance issues"
  rules := ["missing_memory_limits","missing_cpu_limits"]
  
  # EXAMPLE
  msg = sprintf("%s excluded rule because %s", [input.metadata.name, reason])
}

Indeed extending the exception rule with a msg would make sense. I like the idea of having the msg be a form of documentation for the exception. Given how new the exceptions are, I don't mind creating a breaking change, moving from exception[rules] to exception[[rules], msg].

I also have the same requirement where need to justify why I am putting that RULE in exception.

Same here, we have a very large set of manifests being created using kustomize, and having a more sensible output for exceptions will allow users to identify exactly what the exceptions are - is this still something that's likely to be included in conftest?