EPIC Compliance audit checks

[viccuad]

Add background check that inspects all resources in cluster and flags those violating currently enforced policies. For more info, see audit checks RFC and PolicyReport RFC.

For helm charts, kwctl, policy-server, kubewarden-controller(crds), e2e tests, we may want to work against a feature branch instead of merging to main.

Task list:

• [ ] New audit-scanner CLI program and image, in go, that runs the audit algorithm:

  • [x] kubewarden/kubewarden-controller#316

  • [x] kubewarden/audit-scanner#1

  • [ ] audit-scanner - CI/CD Create CD for the audit-scanner, using Github Actions. As examples, see kubewarden/policy-server and kubewarden/kubewarden-controller. As a fresh CD, evaluate if we can use a new approach for this repo.

    This can be done in parallel with stage 1 of the MVP.

    Acceptance criteria:

    • Add CI job with needed linters.
    • Add CD job, that depends on CI job: build and release the audit-scanner container image on amd64, aarm64. This includes SBOM and signing. As for other repos, release tagged images and :latest ones from main.
    • Add security audit job.
    • Add FOSSA scanning job.

  • [x] #336

  • [ ] audit-scanner - MVP Stage 3 - perform evaluations Evolve the audit-scanner to submit fake admission reviews to the Policy Server.

    Acceptance criteria:

    • Fake admission reviews are sent to the right Policy Server, against the right policy
    • Results are collected into a data structure
    • Results are printed to stdout
    • We have unit tests
    • We have integration test if needed

  • [ ] audit-scanner - MVP Stage 4 - create PolicyReport Implement creation of PolicyReport inside of the namespace being analyzed, by using the kube API client and https://github.com/kubernetes-sigs/wg-policy-prototypes.

    Acceptance criteria:

    • See PolicyReport RFC and audit RFC.
    • We have unit tests
    • We have integration test if needed
    • Take care of errors if PolicyReports CRDs are not present in cluster.

  • [ ] audit-scanner - MVP Stage 5.1 - inspect cluster wide resources Evolve the audit-scanner to be able to inspect cluster-wide resources.

    Note: this can be done in parallel of stage 5.2

    Acceptance criteria:

    • The cli allows to either scan a specific namespace, or inspect cluster-wide resources. This depends on how the program is invoked by the user
    • When scanning cluster-wide resources, a ClusterPolicyReport is generated
    • We have unit tests
    • We have integration test if needed

  • [ ] audit-scanner - MVP Stage 5.2 - reuse results from previous evaluations Leverage previous PolicyReport objects as a cache for evaluations

    Note: this can be done in parallel of stage 5.1

    Acceptance criteria:

    • See step 5 of audit RFC
    • We have unit tests
    • We have integration test if needed

  • [ ] audit-scanner - MVP Stage 6 - inspect the whole cluster Change the behavior of the cli app:

    • When no flag are specified -> scan all the Namespaces and the cluster-wide resources
    • When the namespace flag is used: scan only the specified namespace
    • When the cluster-wide flag is used: scan only cluster-wide resources

    During this phase, we can evaluate each namespace in a sequential way (no parallelization).

    Acceptance criteria:

    • When no flag is used -> scan all the Namespaces and the cluster-wide resources
    • When the namespace flag is used: scan only the specified namespace
    • When the cluster-wide flag is used: scan only cluster-wide resources
    • User cannot use the namespace flag and the cluster-wide one at the same time
    • We have unit tests
    • We have integration test if needed

• [ ] policy-server changes:

  • [ ] #337

• [x] kubewarden/kubewarden-controller#318

• [x] kwctl:

  • [x] kubewarden/kwctl#330
  • [x] kubewarden/kwctl#331

• [ ] Helm charts:

  • [ ] Expose audit feature in kubewarden controller helm charts Acceptance criteria:

    • Consume new CRDs with spec.backgroundAudit.
    • Provide Helm values to enable & disable the audit feature, configure the interval of scan
    • By default the audit feature is enabled
    • By default the scan interval is: 60 minutes

  • [ ] Add new metrics to Grafana The grafana dashboard should allow an operator to know how many evaluations are hitting the Policy Servers

  • [ ] Figure out how to distribute PolicyReport CRDs Should the PolicyReport CRDs be shipped by our CRD helm chart? Is there some helm chart already done by the community?

    Acceptance criteria:
    - We know how PolicyReport CRDs are going to be installed on the target cluster
    - Add "rancher annotations" that specify the new dependency on PolicyReports CRD.
    

  • [ ] Release Helm charts with audit feature

• [ ] Update all policies with new spec.backgroundAudit as needed The default value is spec.backgroundAudit is true, hence, we can to set it to false for the policies targeting *. Right now we don't have policies using user/group information.

  Acceptance criteria:

  • All the relevant policies are updated and a new release is done for each one of them

• [ ] Update all policy templates with new spec.backgroundAudit as needed. The default value is spec.backgroundAudit == true. Provide commented metadata and explanation.

• [ ] Write docs on audit feature Acceptance criteria:

  • List new deps (PolicyReport CRDs).
  • How to enable and configure the audit feature.
  • How to interface with PolicyReports (Policy Reporter?).
  • Examples of PolicyReports.
  • What does it mean for policies and policy authors.

• [ ] Add e2e tests for the audit feature Acceptance criteria: Test Helm charts by:

  • Enabling audit feature without needed PolicyReport CRDs (expect failure). - depends on how we plan to distributed the PolicyReport CRDs
  • Enabling audit feature with present PolicyReport CRDs.
  • Trigger Kubernetes CronJob manually and look at expected generated PolicyReports.
  • Enabling Prometheus metrics and OpenTelemetry traces, and check for their output.
  • Disabling audit feature, and seeing that needed CronJob and policy-server changes are made.

• [ ] Release Kubewarden stack Acceptance criteria:

  • Release controller, audit-scanner, policy-server, kwctl, helm-charts, by merging the feat-audit branches into main where needed.
  • Do RC before final release
  • Run e2e tests for stack

• [ ] Write blogpost about new compliance audit checks feature. Acceptance criteria:

  • How it works for end users, how to install and configure.
  • What's new internally, simple explanation of its implementation and how we arrived at it (e.g: point to RFC).
  • What does it mean for policy authors.