[feature]: Support test override files

[micahhausler]

trafficstars

Thanks for this tool! We at AWS have a change we’d like to suggest and would love community input and participation in implementing this.

The CIS Kubernetes Benchmark is a widely recognized guideline for operating and configuring Kubernetes and its various components, but is opinionated in how configuration is set in ways that not all hosted solutions of Kubernetes might agree with.

The proposal I have is for kube-bench is to have a new “config overlay” feature. Taking the node configuration and command as an example, the code currently loads one YAML config file containing the test criteria (ex: node.yaml) and loads how certain tests are scored, the remediation description, and other inputs for the test. In order to allow providers to change certain fields for a test (es: switching “scored” to false, providing a different text description), could kube-bench support a second file that could contain test overrides?

Given the following test file example: (not that there is anything wrong with this specific test)

controls:
version: 1.5
id: 4
text: "Worker Node Security Configuration"
type: "node"
groups:
  # ...
  - id: 4.1
    text: "Worker Node Configuration Files"
    checks:
    - id: 4.1.7
      text: "Ensure that the certificate authorities file permissions are set to 644 or more restrictive (Scored)"
      types: "manual"
      remediation: |
        Run the following command to modify the file permissions of the
        --client-ca-file chmod 644 <filename>
      scored: true

If a provider wanted to set test 4.1.7 to not be scored, and provide reasoning, they could supply the following override file:

controls:
version: 1.5
id: 4
text: "Worker Node Security Configuration"
type: "node"
groups:
  - id: 4.1
    text: "Worker Node Configuration Files"
    checks:
    - id: 4.1.7
      scored: false

and a user would call:

kube-bench node -f node.yaml --override provider.yaml

I think changes to the CIS benchmark are outside the scope of this project, and it is worth it for vendors (AWS included) to work toward refining the common benchmark, but this would be a way for providers to easily iterate and provide their own override files. There could even be an explicit [Override] state for overridden tests to explicitly indicate that the result is overridden

[lizrice]

I’m supportive of this idea. We always envisioned that people might want to edit the test files, but this could make it easier

[leodotcloud]

Currently, we copy over the entire benchmark directory and edit the tests which are not relevant to us and use that with a different name (Reference: https://github.com/rancher/security-scan/tree/master/package/cfg/rke-cis-1.4). I like @micahhausler's idea of being able to override, this would make it easy to track the differences for a given provider.

kube-bench node command accepts -f but kube-bench run doesn't, so I guess we need a common way of overriding for both sub commands.

May be:

kube-bench node --config-dir cis-1.4 -f node.yaml --override-config-dir vendor-cis-1.4

kube-bench run --config-dir cis-1.5 --targets node --override-config-dir vendor-cis-1.4

node.yaml file will be picked up from the override config directory. Either we can have an empty file if there are no overrides for this particular target or a missing file could indicate the same.