[Workload] EPIC: Integrate test cases for kyverno integration

[lixuna]

Is your workload test idea related to a problem? Please describe.

• This issue is for tracking the iterative reviews and merging tests included in PR #1176

Describe the solution you'd like

• The PR will be broken into smaller PRs by the reviewers and merged iteratively

---

Documentation QA:

• [ ] Update installation instructions if needed
• [ ] Update Test Categories md if needed
• [ ] Update USAGE md if needed
  • [ ] How to run
  • [ ] Description and details
    • [ ] What the best practice is
    • [ ] Why are we testing this
  • [ ] Remediation steps if test does not pass

QA tasks

Dev Review:

• [ ] walk through A/C
• [ ] do you get the expected result?
• [ ] if yes,
  • [ ] move to Needs Peer Review column
  • [ ] create Pull Request and follow check list
  • [ ] Assign 1 or more people for peer review
• [ ] if no, document what additional tasks will be needed

Peer review:

• [ ] walk through A/C
• [ ] do you get the expected result?
• [ ] if yes,
  • [ ] move to Reviewer Approved column
  • [ ] Approve pull request
• [ ] if no,
  • [ ] document what did not go as expected, including error messages and screenshots (if possible)
  • [ ] Add comment to pull request
  • [ ] request changes to pull request

[wvwatson]

The Kyverno security tests still need specs @HashNuke The PRs have been merged into main

[HashNuke]

Made some notes on tests in the PR to begin splitting into separate branches.

There are a total of 23 tests. The following have to be done:

1. Pick a test
2. Review code
3. Look for spec test in PR
4. Validate test by running it (may need a good CNF and bad CNF)
5. Create ticket
6. Submit new PR

[Update] Total test count for Kyverno should be 23. Not 24 as mentioned earlier in this comment.

[agentpoyo]

restrict_external_ips was also already merged into main. @HashNuke

[agentpoyo]

We still need spec tests and sample cnf if required done for https://github.com/cncf/cnf-testsuite/issues/1233 https://github.com/cncf/cnf-testsuite/issues/1234 https://github.com/cncf/cnf-testsuite/issues/1235 and https://github.com/cncf/cnf-testsuite/issues/1236

[HashNuke]

@agentpoyo Ah yes. My bad. I missed marking that properly in the sheet above.

1. I'm beginning with adding spec tests for the 4 tests already in main.
2. I'll integrate the other tests once these above is done.

[HashNuke]

[Update] Current output message structure is in the below screenshot. When making updates to container_sock_mounts test, I came across better message in the actual PolicyReport that can be used as remediation message.

It would be better if we use the messages in the PolicyReport since they are more specific. For example, the two validation issues in the screenshot are for the same resource and for the same test. But there are two outputs because one of the volume mount is a docker engine socket. The other volume mount is a containerd engine socket.

[HashNuke]

[Update] Regarding the disallow_helm_tiller test

• Renamed disallow_helm_tiller to helm_tiller
• Moved the test from workload security to platform security (platform:helm_tiller)

[HashNuke]

Tests completed in the kyverno-improvements branch:

• container_sock_mounts
  • Previously called disallow_container_sock_mounts
  • Added to src/tasks/workload/security.cr
• require_labels
  • Added to src/tasks/workload/configuration.cr
• platform:helm_tiller
  • Previously called disallow_helm_tiller
  • Added to src/tasks/platform/security.cr
• external_ips
  • Previously called restrict_external_ips
  • Added to src/tasks/workload/security.cr

[HashNuke]

Added a comment with notes about changing Kyverno to a cli-based utility within the testsuite on another ticket (#1247).

Updated the PR with appropriate changes too (#1266).

[HashNuke]

I reviewed the Kyverno policy files for all tests added in PR #1176. Below are my notes regarding certain tests.

Tests that require discussion

restrict_image_registries (Kyverno policy file)

• The policy file is a sample that has to be updated as per requirement.
• AFAIK this is not something that we can standardise as a guideline. In order to implement this test, the testsuite would have to allow the user to configure registries that can be used.

restrict_volume_types (Kyverno policy file)

• AFAIK this is not something that we can standardise as a guideline. This test would require that the testsuite or the user configure allowed volume types that are allowed.
• Alternative: If we can identify certain volume types that the CNF should not be using, then we can use this test to check if the volume types are not used.

add_network_policy (Kyverno Policy file)

• The policy file "adds" a default network policy for namespaces. It does not test for anything.
• We can see if this test can be changed to having a default network policy for namespaces. I've also created a ticket for default_network_policy #1271. But looks like network policies are based on pod selectors. So we'll have to look for a network policy that matches something like the example in the Kubernetes docs.

Overlapping tests

The following tests in the PR are overlapping with existing tests in the testsuite.

1. deny_privilege_escalation
   • Kyverno Policy file
   • This is the same as the kubescape-based test privilege_escalation
2. disallow_add_capabilities
   • Kyverno Policy file
   • Overlaps with two Kubescape-based tests dangerous_capabilities and insecure_capabilities
3. disallow_host_namespaces
   • Kyverno Policy file
   • Overlaps with two kubescape-based tests host_network and host_pid_ipc_privileges
4. disallow_host_path
   • Kyverno policy file
   • Overlaps with Kubescape-based hostpath_mounts test.
5. disallow_host_ports
   • Kyverno Policy file
   • Overlaps with hostport_not_used test.
6. disallow_privileged_containers
   • Kyverno Policy file
   • Overlaps with privileged_containers test.
7. require_non_root_groups
   • The Kyverno policy file url used in the PR does not exist in the repo anymore. This might be the new Kyverno policy file
   • Overlaps with existing non_root_containers test.
8. require_requests_limits
   • Kyverno policy file
   • Overlaps with existing Kubescape-based resource_policies test.
9. require_ro_rootfs
   • Kyverno Policy file
   • Overlaps with immutable_file_systems test.
10. restrict_node_port
    • Kyverno policy file
    • Overlaps with nodeport_not_used test.
11. run_as_non_root
    • Kyverno policy file
    • Overlaps with non_root_containers test.
12. pod_probes
    • Kyverno policy file
    • Overlaps with liveness and readiness tests. These two tests are listed in the resilience category in the points.yml file, but are in the reliability.cr file. They are also not configured to run when the reliability or resilience test categories are run.

[lixuna]

Hi @HashNuke Can this issue be closed?