feat(policies): add rego boilerplate injection

[Piskoo]

Summary

Implemented automatic missing boilerplate injection for rego policies in the engine before policy evaluation.

Changes

• Rego policy engine now detects missing required rules and injects them after package and import declarations.
• Removed boilerplate rules from policy template used in policy devel init.
• Updated policy devel lint since we no longer require result rule to be present in the policy.

Example

Before policy required declaration of all rules used internally

package main

import rego.v1

result := {
    "skipped": skipped,
    "violations": violations,
    "skip_reason": skip_reason,
}

default skip_reason := ""

skip_reason := m if {
    not valid_input
    m := "the file content is not recognized"
}

default skipped := true

skipped := false if valid_input

default valid_input := true

violations contains msg if {
    count(input.components) < 2
    msg := "SBOM must have at least 2 components"
}

Now these rules are injected into policy if they are missing, so above policy can be simplified to:

package main

import rego.v1

violations contains msg if {
    count(input.components) < 2
    msg := "SBOM must have at least 2 components"
}

[Piskoo]

Overall, it looks good to me, and I don’t want to block the PR. However, I have a question: instead of injecting the template into the incoming policy source, why don’t we create an external rego package that we automatically import into the file? Then, both the template and the package would be loaded into the registry engine.

I went with injecting the template mainly to keep the setup straightforward and avoid changing how policies are loaded right now. Providing it in another package would require policy authors to refer to these rules by a namespace, which would change how existing policies are written and potentially break compatibility with the current structure.

[migmartri]

Overall, it looks good to me, and I don’t want to block the PR. However, I have a question: instead of injecting the template into the incoming policy source, why don’t we create an external rego package that we automatically import into the file? Then, both the template and the package would be loaded into the registry engine.

@javirln has a very good point here, what's the best practice in general w.r.t extensibility of rego code? i.e OPA SDK

Providing it in another package would require policy authors to refer to these rules by a namespace

right, but we could migrate little by little now? I am not saying we should do it like this, I am more interested in knowing what's the idiomatic way of doing this, specially since we'll be providing an SDK methods.

[Piskoo]

right, but we could migrate little by little now? I am not saying we should do it like this, I am more interested in knowing what's the idiomatic way of doing this, specially since we'll be providing an SDK methods.

The idiomatic OPA way is like @javirln mentioned to define shared helpers in separate packages and import them, since that supports versioning and reuse.

[jiparis]

right, but we could migrate little by little now? I am not saying we should do it like this, I am more interested in knowing what's the idiomatic way of doing this, specially since we'll be providing an SDK methods.

The idiomatic OPA way is like @javirln mentioned to define shared helpers in separate packages and import them, since that supports versioning and reuse.

There are two different topics mentioned in these comments:

• one is avoiding boilerplate. The proposal from @javirln is importing external rules using the idiomatic import keyword. Note that this doesn't expose those rules to the caller, but just makes them available in the current script. So, let's say we have something like this:

# common.rego
result := {
	"skipped": skipped,
	"violations": violations,
	"skip_reason": skip_reason,
}

In the policy we would inject something like this (because we want to keep it simple for users so that they don't worry about boilerplate):

import data.common

result := common.result

As you can see, we are not improving the UX at all, just replacing one rule by an import and another rule.

• the second topic mentioned here is injecting functions into the policy. This is completely out of scope for this issue and PR. It's fairly well documented and could be addressed in another feature.

[migmartri]

right, but we could migrate little by little now? I am not saying we should do it like this, I am more interested in knowing what's the idiomatic way of doing this, specially since we'll be providing an SDK methods.

The idiomatic OPA way is like @javirln mentioned to define shared helpers in separate packages and import them, since that supports versioning and reuse.

There are two different topics mentioned in these comments:

• one is avoiding boilerplate. The proposal from @javirln is importing external rules using the idiomatic import keyword. Note that this doesn't expose those rules to the caller, but just makes them available in the current script. So, let's say we have something like this:

# common.rego
result := {
	"skipped": skipped,
	"violations": violations,
	"skip_reason": skip_reason,
}

In the policy we would inject something like this (because we want to keep it simple for users so that they don't worry about boilerplate):

import data.common

result := common.result

As you can see, we are not improving the UX at all, just replacing one rule by an import and another rule.

• the second topic mentioned here is injecting functions into the policy. This is completely out of scope for this issue and PR. It's fairly well documented and could be addressed in another feature.

Thanks for the breakdown, I understand now, the current approach LGTM!