KubeArmor icon indicating copy to clipboard operation
KubeArmor copied to clipboard

Improve system test coverage and practices for KubeArmor

Open DelusionalOptimist opened this issue 1 year ago • 4 comments

Description

Currently our Ginkgo based test suite for running system tests is lacking behind. We are not testing all of KubeArmor's features in different important supported environments.

Goals

The main goals of this are going to be as given below, please see attached issues for more details

  • [ ] Calculate code coverage in CI - https://github.com/kubearmor/KubeArmor/issues/845
  • [ ] Add test suite for host protection capabilities of KubeArmor in Kubernetes and Non-Kubernetes mode - https://github.com/kubearmor/KubeArmor/issues/1625
  • [ ] Extend existing container protection test suite to Non-Kubernetes mode
  • [ ] Test on ARM based machines - https://github.com/kubearmor/KubeArmor/issues/1228
  • [ ] Automate running post-release tests for certain, frequently used environments - https://github.com/kubearmor/KubeArmor/issues/1675
  • [ ] Improvise the existing test suites -
    • [ ] Reduce completion time (30 minutes for each enforcer/runtime currently)
      • [ ] Using a common workflow for building images
      • [ ] Only build the component which actually changed
    • [ ] Reduce flakiness
    • [ ] Suggestions welcome...
  • [ ] Improve docs on writing tests

Extended Goals (to be pursued only if enough time left)

  • [ ] Understand and implement fuzz testing in context of KubeArmor (any one of the modules)

Test Matrix Tracker

Provider Environment Type AppArmor BPF LSM Arch Runtime Frequency
Onprem k3s Kubernetes Container :heavy_check_mark:, Host :x: Container :heavy_check_mark:, Host :x: amd64 :heavy_check_mark:, arm :x: Docker :heavy_check_mark: , Containerd :heavy_check_mark:, CRI-O :heavy_check_mark:, Push, PRs, Release
Linux Ubuntu, Fedora Non-k8s Container :x:, Host :x: Container :x:, Host :x: amd64 :x: Docker :x: , Systemd :x: Push, PRs, Release
Onprem KinD/Minikube, kubeadm Kubernetes Container :x:, Host :x: Container :x:, Host :x: amd64 :x: Virtualbox :x:, Containerd :x: Release
Managed TBD Kubernetes Container :x:, Host :x: Container :x:, Host :x: amd64 :x: arm64 :x: TBD Release

NOTES

  • Non-k8s test suite would work with both KubeArmor running in container mode as well as systemd mode.
  • For each of the above testing of the below aspects will be considered to mark it as complete
    • [ ] Observability
    • [ ] Enforcement
    • [ ] Alerts

Prerequisite Task

Create at least one new and unique test case for KubeArmor (can be in any mode) using the ginkgo framework and show the total improvement in coverage. The test coverage tool doesn't have to be run as part of the CI.

Deadline - 31 May 2024 11:59 PM UTC

Please don't share your task solutions publicly, prefer e-mailing a link to your fork/branch to mentors.

References

DelusionalOptimist avatar May 08 '24 08:05 DelusionalOptimist

are there any pretasks?

cc: @DelusionalOptimist

officialasishkumar avatar May 11 '24 07:05 officialasishkumar

@DelusionalOptimist sir is there any pretask?

Ayush9026 avatar May 12 '24 10:05 Ayush9026

Hey @officialasishkumar @Ayush9026, the issue description has been updated with the prerequisite task.

DelusionalOptimist avatar May 14 '24 15:05 DelusionalOptimist

Untitled-2024-01-03-1351(4)

// @navin772

daemon1024 avatar Jul 08 '24 11:07 daemon1024