gatekeeper Provide a metric with OPA runtime errors

Describe the solution you'd like I'd like to have a metric to monitor gatekeeper having OPA Runtime errors like this one:

libs[\"lib_1\"]:45: eval_conflict_error: functions must not produce multiple outputs for same inputs

Currently there seems to be no way to differentiate between evaluations which returned a violation and evaluations which ended with a runtime error -- both are considered a violation.

Anything else you would like to add: It might be a simple counter, probably labeled by enforcement action both audit and webhook should have it

Environment:

Gatekeeper version: v3.11.0

May 02 '23 16:05 stek29

Hi,

Thanks for the suggestion!

I agree that we should have a metric that shows runtime errors when evaluating rego.

However, doing so will require some work as the Constraint Framework would need to be modified in order for it to know how to which Constraint Template failed (currently it will just return a runtime error that is not specific to particular constraint).

The next related, but technically orthogonal, problem would be to figure out how to give the user control over the enforcement action when these runtime errors are encountered. At first glance, we could expose an extra field in a Constraint to specify what enforcement action should be taken on runtime failure, maybe something like spec.failureEnforcementAction (name TBD).