Refactor compliance tests from YAML to Rego

[anderseknert]

YAML might have been the best choice at the time, but now that metadata is easily represented in Rego, we should have our compliance tests be Rego files rather than Rego embedded in YAML docs. Working with Rego directly means we can have syntax highlighting, debugging, linting, editor support, and all the other niceties we now have for Rego authoring.

Taking the first test I found as an example, I suggest that instead of this:

---
cases:
  - note: aggregates/count
    query: data.generated.p = x
    modules:
      - |
        package generated

        p contains x if {
        	__local0__ = data.a
        	count(__local0__, x)
        }
    data:
      a:
        - 1
        - 2
        - 3
        - 4
    want_result:
      - x:
          - 4
    sort_bindings: true

We'd do this:

# METADATA
# title: aggregates/count
# custom:
#   query: data.generated.p = x
#   data: {a: [1, 2, 3, 4]}
#   want_result: [{x: [4]}]
package generated

p contains x if {
	__local0__ = data.a
	count(__local0__, x)
}

Metadata comments are of course just YAML too, so the query/data/input attributes can remain as such, just embedded in the policy, and parsed by the test runner.

I'm sure there are some details we'd have to give some more thought to, but posting this here to start somewhere.

[anderseknert]

I suppose one alternative to having input and data embedded would be to have one directory per test, and an input.json file for the input and data.json (or data.yaml) for data. This would be nice in that it would make it very easy to evaluate any test using just opa eval or in an editor in that directory. The downside would obviously be that you'd need to check more than one file to understand what the inputs for a test are.

[stale[bot]]

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days. Although currently inactive, the issue could still be considered and actively worked on in the future. More details about the use-case this issue attempts to address, the value provided by completing it or possible solutions to resolve it would help to prioritize the issue.