feat: kyverno policies

[srodenhuis]

This MVP includes:

• adding Kyverno as a core app with 2 deployment profiles (DevTest and Prod)
• offer 5 policy profiles with (to start with) 25 policies
• policies are per team
• teams can customize policies, deactivate them or change the default action (enforce, audit)
• admins can block teams from changing policies

[gitguardian[bot]]

⚠️ GitGuardian has uncovered 4 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
9221902	Triggered	Username Password	13c6786d6693cc6a7517b0978178c1e5b1697819	charts/kyverno/ci/imagePullSecrets-values.yaml	View secret
9221902	Triggered	Username Password	eba90fc93efc6272257f64ece0b00c6f5c1d5f11	charts/kyverno/ci/imagePullSecrets-values.yaml	View secret
9221902	Triggered	Username Password	fd8ac96ed5bf4dc8ba681d96c15ef6beccbe55e3	charts/kyverno/ci/imagePullSecrets-values.yaml	View secret
9221902	Triggered	Username Password	e81d49dcdddf730fd5ed77d6a1444ac2ca171414	charts/kyverno/values.yaml	View secret

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code.
2. Replace and store your secrets safely. Learn here the best practices.
3. Revoke and rotate these secrets.
4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

• following these best practices for managing and storing secrets including API keys and other credentials
• install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

---

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}

[CasLubbers]

One more question, maybe I cannot find it because the PR is so big. But what are the default policies we set for the team-admin namespace?

[merll]

I have attempted but failed to deploy this app. Some pods fail to start:

kyverno/kyverno-admission-controller

2024-04-17T08:13:09Z    ERROR    klog    cache/reflector.go:147    k8s.io/[email protected]/tools/cache/reflector.go:229: Failed to watch *v2alpha1.GlobalContextEntry: failed to list *v2alpha1.GlobalContextEntry: the server could not find the requested resource (get globalcontextentries.kyverno.io)
2024-04-17T08:13:31Z    INFO    klog    cache/reflector.go:539    k8s.io/[email protected]/tools/cache/reflector.go:229: failed to list *v2alpha1.GlobalContextEntry: the server could not find the requested resource (get globalcontextentries.kyverno.io)

kyverno/kyverno-cleanup-controller

2024-04-17T08:11:43Z    ERROR    setup    cleanup-controller/main.go:124    sanity checks failed    {"error": "failed to check CRD cleanuppolicies.kyverno.io is installed: customresourcedefinitions.apiextensions.k8s.io \"cleanuppolicies.kyverno.io\" is forbidden: User \"system:serviceaccount:kyverno:kyverno-cleanup-controller\" cannot get resource \"customresourcedefinitions\" in API group \"apiextensions.k8s.io\" at the cluster scope; failed to check CRD clustercleanuppolicies.kyverno.io is installed: customresourcedefinitions.apiextensions.k8s.io \"clustercleanuppolicies.kyverno.io\" is forbidden: User \"system:serviceaccount:kyverno:kyverno-cleanup-controller\" cannot get resource \"customresourcedefinitions\" in API group \"apiextensions.k8s.io\" at the cluster scope", "errorCauses": [{"error": "failed to check CRD cleanuppolicies.kyverno.io is installed: customresourcedefinitions.apiextensions.k8s.io \"cleanuppolicies.kyverno.io\" is forbidden: User \"system:serviceaccount:kyverno:kyverno-cleanup-controller\" cannot get resource \"customresourcedefinitions\" in API group \"apiextensions.k8s.io\" at the cluster scope"}, {"error": "failed to check CRD clustercleanuppolicies.kyverno.io is installed: customresourcedefinitions.apiextensions.k8s.io \"clustercleanuppolicies.kyverno.io\" is forbidden: User \"system:serviceaccount:kyverno:kyverno-cleanup-controller\" cannot get resource \"customresourcedefinitions\" in API group \"apiextensions.k8s.io\" at the cluster scope"}]}

Any ideas?

[merll]

@ferruhcihan gave me the hint that the CRD for globalcontextentries.kyverno.io is missing. It is not part of the helm chart (which I am guessing is based on the 1.10 release of Kyverno – the 1.12 release branch has the CRD, stable is 1.11 which does not have it), but somehow it seems to deploy a newer app version that relies on it.

Possibly the helm chart used was from the main branch at the time. The release helm charts include a specific appVersion to avoid this issue.

[j-zimnowoda]

Some things I found which can be removed I think:

• The file bin/build-gatekeeper-artifacts.sh can also be removed right?
• In the README.md line 133 OPA/Gatekeeper... can be removed right?
• The policies directory can also be removed right? If so the project structure in docs/development.md should be adjusted corectly.
• The checkPolicies command in the Otomi CLI can also be removed right? This requires changes in index.ts, binzx/README.md, check-policies.ts, tests.ts
• The file docs/policies.md should be updated or removed right?
• In the file src/cmd/destroy.ts on line 105 gatekeeper.sh can be removed right?
• In the file src/cmd/test.ts on line 21 if (!values?.apps.gatekeeper!.disableValidatingWebhook) await checkPolicies() can be removed right?
• In the file tests/integration/upgrade.yaml on line 18 gatekeeper can be removed right?

Good points! We are going to perform further refactoring after this PR I merged.