chainloop icon indicating copy to clipboard operation
chainloop copied to clipboard

Improve policy results format

Open jiparis opened this issue 1 year ago • 0 comments

Currently, policy evaluations are reported as a single violations rule result, that will contain elements in the case of non compliance. However, if the policy couldn't be evaluated (because of invalid input or some conditions were not met) it would still report an empty violations set, which is wrong, since it couldn't be evaluated. There are three variables in policy evaluations:

  • does it need to be evaluated? : we know this because of the policy selection logic (by kind and/or material name)
  • was it evaluated?: this would come from the Rego itself
  • violations

The proposal is to add a structured output to policies, leaving room for future improvements:

{
    "result": {
        "evaluated": true,
        "violations": []
    },
    "violations": [] // <-- current policy output
  }

How evaluated is generated, it's up to each policy implementation. For example, a CycloneDX policy would set it if there are components in the report, or a SARIF policy for trivy would set it if runs[].tool.driver.name == "Trivy".

jiparis avatar Sep 13 '24 12:09 jiparis