Add a --fail-defined flag to opa exec

[phantlantis]

What is the underlying problem you're trying to solve?

I see that there is an option in opa eval that tells opa to return a nonzero exit code if anything comes back defined in the result[] block of a policy evaluation - referred to as --fail-defined. When attempting to run opa exec with this flag, however, I receive an error that the flag is not recognized. Because of this, when I run opa exec and receive defined results for a deny decision, the CLI returns a zero exit code which creates extra work to orchestrate with tools that are dependent on CLI exit codes for decisions.

Describe the ideal solution

Ideally, implementing the --fail-defined flag in opa exec so that if something comes back defined in a result[] block, then a nonzero exit code is returned via CLI.

Describe a "Good Enough" solution

Some additional documentation examples of how to achieve equivalent functionality in opa eval would also probably be sufficient.

Additional Context

[stale[bot]]

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days.

[byronic]

Yeah, this is pretty important. See from the output here that both decisions exit with a 0 code, making this useless for a pipeline or automated run:

$ opa exec --decision terraform/analysis/authz --bundle ./policy plan.json
{
  "result": [
    {
      "path": "plan.json",
      "result": true
    }
  ]
}
$ echo $?
0
$ opa exec --decision terraform/analysis/authz --bundle ./policy plan.json
{
  "result": [
    {
      "path": "plan.json",
      "result": false
    }
  ]
}
$ echo $?
0

Given that it's a feature I need, I'm going to get started on it.

[anderseknert]

Great! I'll assign you to the issue then 👍

[byronic]

Thanks much! I am in-progress on this, currently troubleshooting the build. Hopefully I'll have something to show by the end of the week

[anderseknert]

No stress! If you need any help getting things set up, the #contributors channel in the OPA Slack is a great place to ask questions 😃

[byronic]

My first pass at this is done, but I expect some changes to come up on review. Sample output of the current PR #5191 :

$ opa exec --decision terraform/analysis/authz --fail-defined --bundle ./policy plan.json
{
  "result": [
    {
      "path": "plan.json",
      "result": true
    }
  ]
}

$ echo $?
0

$ opa exec --decision terraform/analysis/authz --fail-defined --bundle ./policy plan.json
{
  "result": [
    {
      "path": "plan.json",
      "result": false
    }
  ]
}
{"err":"exec error: there were 1 failures and 0 errors counted in the results list, and --fail-defined is set","level":"error","msg":"Unexpected error.","time":"2022-09-27T22:31:31-05:00"}

$ echo $?
1

[byronic]

The related PR has been updated to exit non-zero on any defined result (or error), and zero on undefined results w/ no errors.

So the above comment would show nonzero exit codes for both false and true, but as noted in the PR comment you can condense a truthy value into an undefined result if needed at the policy level -- cf. https://github.com/open-policy-agent/opa/pull/5191#issuecomment-1261105329

[byronic]

This is still in-progress, now with a fresh PR #5295.