trivy feat: add k8s components

Signed-off-by: Jose Donizetti [email protected]

Description

Adds a new table with checks for infra assesment.

trivy k8s all --report=summary --namespace=kube-system
34 / 34 [--------------------------------------------------------------------------------------------------------------------] 100.00% 1 p/s

Summary Report for minikube
┌─────────────┬──────────────────────────────────────┬─────────────────────┬────────────────────┬───────────────────┐
│  Namespace  │               Resource               │   Vulnerabilities   │ Misconfigurations  │      Secrets      │
│             │                                      ├───┬────┬───┬────┬───┼───┬───┬───┬────┬───┼───┬───┬───┬───┬───┤
│             │                                      │ C │ H  │ M │ L  │ U │ C │ H │ M │ L  │ U │ C │ H │ M │ L │ U │
├─────────────┼──────────────────────────────────────┼───┼────┼───┼────┼───┼───┼───┼───┼────┼───┼───┼───┼───┼───┼───┤
│ kube-system │ Deployment/coredns                   │ 1 │ 2  │ 1 │ 1  │ 4 │   │   │ 3 │ 5  │   │   │   │   │   │   │
│ kube-system │ Pod/etcd-minikube                    │   │ 16 │ 4 │    │ 4 │   │ 1 │ 3 │ 7  │   │   │   │   │   │   │
│ kube-system │ Pod/kube-scheduler-minikube          │   │ 1  │   │    │   │   │ 1 │ 3 │ 8  │   │   │   │   │   │   │
│ kube-system │ Pod/storage-provisioner              │   │ 8  │ 2 │    │ 2 │   │ 1 │ 5 │ 10 │   │   │   │   │   │   │
│ kube-system │ DaemonSet/kube-proxy                 │   │ 2  │ 2 │ 22 │   │   │ 2 │ 4 │ 10 │   │   │   │   │   │   │
│ kube-system │ Pod/kube-controller-manager-minikube │   │ 1  │ 1 │ 1  │ 2 │   │ 1 │ 3 │ 8  │   │   │   │   │   │   │
│ kube-system │ Service/kube-dns                     │   │    │   │    │   │   │   │ 1 │    │   │   │   │   │   │   │
│ kube-system │ Pod/kube-apiserver-minikube          │   │ 1  │ 1 │ 1  │ 2 │   │ 1 │ 3 │ 9  │   │   │   │   │   │   │
└─────────────┴──────────────────────────────────────┴───┴────┴───┴────┴───┴───┴───┴───┴────┴───┴───┴───┴───┴───┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN


Summary Report for minikube
┌─────────────┬─────────────────────────────────────────────────────┬───────────────────┐
│  Namespace  │                      Resource                       │  RBAC Assessment  │
│             │                                                     ├───┬───┬───┬───┬───┤
│             │                                                     │ C │ H │ M │ L │ U │
├─────────────┼─────────────────────────────────────────────────────┼───┼───┼───┼───┼───┤
│ kube-system │ Role/system::leader-locking-kube-controller-manager │   │   │ 1 │   │   │
│ kube-system │ Role/system:controller:bootstrap-signer             │ 1 │   │   │   │   │
│ kube-system │ Role/system:controller:cloud-provider               │   │   │ 1 │   │   │
│ kube-system │ Role/system:controller:token-cleaner                │ 1 │   │   │   │   │
│ kube-system │ Role/system:persistent-volume-provisioner           │   │ 2 │   │   │   │
│ kube-system │ Role/system::leader-locking-kube-scheduler          │   │   │ 1 │   │   │
└─────────────┴─────────────────────────────────────────────────────┴───┴───┴───┴───┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN


Summary Report for minikube
┌─────────────┬──────────────────────────────────────┬─────────────────────────────┐
│  Namespace  │               Resource               │ Kubernetes Infra Assessment │
│             │                                      ├────┬────┬────┬─────┬────────┤
│             │                                      │ C  │ H  │ M  │ L   │   U    │
├─────────────┼──────────────────────────────────────┼────┼────┼────┼─────┼────────┤
│ kube-system │ Pod/kube-apiserver-minikube          │    │    │ 1  │ 10  │        │
│ kube-system │ Pod/kube-controller-manager-minikube │    │    │    │ 3   │        │
│ kube-system │ Pod/kube-scheduler-minikube          │    │    │    │ 1   │        │
└─────────────┴──────────────────────────────────────┴────┴────┴────┴─────┴────────┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN

filtering by workload components:

trivy k8s all --report=summary --namespace=kube-system --security-checks=config --components=workload
34 / 34 [--------------------------------------------------------------------------------------------------------------------] 100.00% 4 p/s

Summary Report for minikube
┌─────────────┬──────────────────────────────────────┬────────────────────┐
│  Namespace  │               Resource               │ Misconfigurations  │
│             │                                      ├───┬───┬───┬────┬───┤
│             │                                      │ C │ H │ M │ L  │ U │
├─────────────┼──────────────────────────────────────┼───┼───┼───┼────┼───┤
│ kube-system │ Service/kube-dns                     │   │   │ 1 │    │   │
│ kube-system │ Pod/kube-apiserver-minikube          │   │ 1 │ 3 │ 9  │   │
│ kube-system │ Pod/kube-scheduler-minikube          │   │ 1 │ 3 │ 8  │   │
│ kube-system │ Pod/storage-provisioner              │   │ 1 │ 5 │ 10 │   │
│ kube-system │ Pod/etcd-minikube                    │   │ 1 │ 3 │ 7  │   │
│ kube-system │ Deployment/coredns                   │   │   │ 3 │ 5  │   │
│ kube-system │ DaemonSet/kube-proxy                 │   │ 2 │ 4 │ 10 │   │
│ kube-system │ Pod/kube-controller-manager-minikube │   │ 1 │ 3 │ 8  │   │
└─────────────┴──────────────────────────────────────┴───┴───┴───┴────┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN

filtering by infra components:

trivy k8s all --report=summary --namespace=kube-system --security-checks=config --components=infra
34 / 34 [--------------------------------------------------------------------------------------------------------------------] 100.00% 4 p/s

Summary Report for minikube
┌─────────────┬──────────────────────────────────────┬─────────────────────────────┐
│  Namespace  │               Resource               │ Kubernetes Infra Assessment │
│             │                                      ├────┬────┬────┬─────┬────────┤
│             │                                      │ C  │ H  │ M  │ L   │   U    │
├─────────────┼──────────────────────────────────────┼────┼────┼────┼─────┼────────┤
│ kube-system │ Pod/kube-scheduler-minikube          │    │    │    │ 1   │        │
│ kube-system │ Pod/kube-apiserver-minikube          │    │    │ 1  │ 10  │        │
│ kube-system │ Pod/kube-controller-manager-minikube │    │    │    │ 3   │        │
└─────────────┴──────────────────────────────────────┴────┴────┴────┴─────┴────────┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN

Related issues

Close https://github.com/aquasecurity/trivy/issues/2766

Remove this section if you don't have related PRs.

Checklist

[x] I've read the guidelines for contributing to this repository.
[x] I've followed the conventions in the PR title.
[x] I've added tests that prove my fix is effective or that my feature works.
[x] I've updated the documentation with the relevant information (if needed).
[x] I've added usage information (if the PR introduces new options)
[x] I've included a "before" and "after" example to the description (if the PR is a user interface change).

Jul 26 '22 00:07 josedonizetti

@chen-keinan Can you please review it?

Aug 17 '22 13:08 knqyf263

@knqyf263 @chen-keinan I'm changing it currently, will be ready for review soon.

Aug 17 '22 13:08 josedonizetti

All committers have signed the CLA.

Aug 30 '22 07:08 CLAassistant

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

Aug 30 '22 07:08 CLAassistant

@chen-keinan can you review it? pls

Aug 30 '22 07:08 josedonizetti

@josedonizetti Look Good! added few comments

Aug 31 '22 08:08 chen-keinan

@josedonizetti could you please relate issue to it.

when running this (scan cluster for vuln only) command trivy k8s --security-checks vuln --report summary cluster I get no result. 164 / 164 [----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 781 p/s however when running the full cluster command I get complete result : k8s --report summary cluster

Summary Report for rancher-desktop
┌──────────────┬───────────────────────────────────┬──────────────────────────────┬────────────────────┬───────────────────┐
│  Namespace   │             Resource              │       Vulnerabilities        │ Misconfigurations  │      Secrets      │
│              │                                   ├─────┬──────┬──────┬─────┬────┼───┬───┬───┬────┬───┼───┬───┬───┬───┬───┤
│              │                                   │  C  │  H   │  M   │  L  │ U  │ C │ H │ M │ L  │ U │ C │ H │ M │ L │ U │
├──────────────┼───────────────────────────────────┼─────┼──────┼──────┼─────┼────┼───┼───┼───┼────┼───┼───┼───┼───┼───┼───┤
│ trivy-system │ Job/scan-vulnerabilityreport      │     │      │  1   │ 1   │ 2  │   │   │ 3 │ 4  │   │   │   │   │   │   │
│ profefe      │ Deployment/profefe                │  4  │  29  │  10  │ 2   │    │   │   │ 2 │ 10 │   │   │   │   │   │   │
│ kube-system  │ Deployment/metrics-server         │     │      │      │     │ 6  │   │   │ 3 │ 10 │   │   │   │   │   │   │
│ kube-system  │ Service/metrics-server            │     │      │      │     │    │   │   │ 1 │    │   │   │   │   │   │   │
│ kube-system  │ Deployment/coredns                │     │  7   │  6   │ 1   │ 5  │   │   │ 3 │ 5  │   │   │   │   │   │   │
│ kube-system  │ Deployment/local-path-provisioner │  4  │  32  │  10  │ 2   │    │   │   │ 3 │ 10 │   │   │   │   │   │   │
│ kube-system  │ Service/kube-dns                  │     │      │      │     │    │   │   │ 1 │    │   │   │   │   │   │   │
│ default      │ ReplicaSet/frontend               │ 565 │ 1372 │ 1157 │ 894 │ 64 │   │   │ 2 │ 10 │   │   │   │   │   │   │
│ default      │ Job/pi                            │ 61  │ 434  │ 368  │ 566 │ 4  │   │   │ 2 │ 10 │   │   │   │   │   │   │
└──────────────┴───────────────────────────────────┴─────┴──────┴──────┴─────┴────┴───┴───┴───┴────┴───┴───┴───┴───┴───┴───┘

Sep 01 '22 15:09 chen-keinan

@chen-keinan Can you give it another try? I increased the report test coverage from 33% to 79%, which caught a few bugs including the one you mentioned above.

Sep 03 '22 01:09 josedonizetti

LGTM 🚀 I would check if needed to update some documentation , example

Sep 04 '22 07:09 chen-keinan

@chen-keinan @knqyf263 can you review? Did the change to support --components. If you agree with the design/code I'll add documentation next (maybe in a follow up PR), just to be sure this is what we want before going deeper.

Sep 18 '22 02:09 josedonizetti

@chen-keinan @knqyf263 I really would like this to land on the next release. Let me know if anything is missing for the review.

Oct 06 '22 18:10 josedonizetti

@josedonizetti looks good 🚀 , there is something strange experience wise when we run filter by workload we still get rbac results :

trivy k8s all --namespace=kube-system  --report=summary --components=workload
34 / 34 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 2 p/s

Summary Report for minikube
┌─────────────┬──────────────────────────────────────┬─────────────────────┬────────────────────┬───────────────────┐
│  Namespace  │               Resource               │   Vulnerabilities   │ Misconfigurations  │      Secrets      │
│             │                                      ├───┬────┬───┬────┬───┼───┬───┬───┬────┬───┼───┬───┬───┬───┬───┤
│             │                                      │ C │ H  │ M │ L  │ U │ C │ H │ M │ L  │ U │ C │ H │ M │ L │ U │
├─────────────┼──────────────────────────────────────┼───┼────┼───┼────┼───┼───┼───┼───┼────┼───┼───┼───┼───┼───┼───┤
│ kube-system │ Pod/etcd-minikube                    │   │ 16 │ 4 │    │ 4 │   │ 1 │ 3 │ 7  │   │   │   │   │   │   │
│ kube-system │ Pod/kube-apiserver-minikube          │   │    │   │    │   │   │ 1 │ 3 │ 9  │   │   │   │   │   │   │
│ kube-system │ Pod/kube-scheduler-minikube          │   │    │   │    │   │   │ 1 │ 3 │ 8  │   │   │   │   │   │   │
│ kube-system │ Deployment/coredns                   │   │ 6  │ 2 │ 1  │ 5 │   │   │ 3 │ 5  │   │   │   │   │   │   │
│ kube-system │ DaemonSet/kube-proxy                 │ 7 │ 11 │ 3 │ 56 │   │   │ 2 │ 4 │ 10 │   │   │   │   │   │   │
│ kube-system │ Service/kube-dns                     │   │    │   │    │   │   │   │ 1 │    │   │   │   │   │   │   │
│ kube-system │ Pod/kube-controller-manager-minikube │   │    │   │    │   │   │ 1 │ 3 │ 8  │   │   │   │   │   │   │
│ kube-system │ Pod/storage-provisioner              │   │ 8  │ 2 │    │ 3 │   │ 1 │ 5 │ 10 │   │   │   │   │   │   │
└─────────────┴──────────────────────────────────────┴───┴────┴───┴────┴───┴───┴───┴───┴────┴───┴───┴───┴───┴───┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN


Summary Report for minikube
┌─────────────┬─────────────────────────────────────────────────────┬───────────────────┐
│  Namespace  │                      Resource                       │  RBAC Assessment  │
│             │                                                     ├───┬───┬───┬───┬───┤
│             │                                                     │ C │ H │ M │ L │ U │
├─────────────┼─────────────────────────────────────────────────────┼───┼───┼───┼───┼───┤
│ kube-system │ Role/system::leader-locking-kube-scheduler          │   │   │ 1 │   │   │
│ kube-system │ Role/system:controller:cloud-provider               │   │   │ 1 │   │   │
│ kube-system │ Role/system:controller:bootstrap-signer             │ 1 │   │   │   │   │
│ kube-system │ Role/system:persistent-volume-provisioner           │   │ 2 │   │   │   │
│ kube-system │ Role/system::leader-locking-kube-controller-manager │   │   │ 1 │   │   │
│ kube-system │ Role/system:controller:token-cleaner                │ 1 │   │   │   │   │
└─────────────┴─────────────────────────────────────────────────────┴───┴───┴───┴───┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN

Oct 11 '22 07:10 chen-keinan

is the --components=infra / --components=workload is not supported on specific resource scan ? trivy k8s Pod/kube-apiserver-minikube --namespace=kube-system --components=infra

result :

... FATAL unknown flag: --components

Oct 11 '22 07:10 chen-keinan

is the --components=infra / --components=workload is not supported on specific resource scan ? trivy k8s Pod/kube-apiserver-minikube --namespace=kube-system --components=infra

result :

... FATAL unknown flag: --components

@chen-keinan are you sure you had this branch compiled when you executed this command? Because it is saying the flag doesn't exist, but the flag is global for trivy k8s. Testing locally now, I see:

./trivy k8s -n kube-system --components=workload pod/kube-apiserver-minikube | grep See
1 / 1 [------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 0 p/s
See https://avd.aquasec.com/misconfig/ksv0012
See https://avd.aquasec.com/misconfig/ksv001
See https://avd.aquasec.com/misconfig/ksv003
See https://avd.aquasec.com/misconfig/ksv009
See https://avd.aquasec.com/misconfig/ksv011
See https://avd.aquasec.com/misconfig/ksv012
See https://avd.aquasec.com/misconfig/ksv014
See https://avd.aquasec.com/misconfig/ksv016
See https://avd.aquasec.com/misconfig/ksv018
See https://avd.aquasec.com/misconfig/ksv020
See https://avd.aquasec.com/misconfig/ksv021
See https://avd.aquasec.com/misconfig/ksv023
See https://avd.aquasec.com/misconfig/ksv106

trivy k8s -n kube-system --components=infra pod/kube-apiserver-minikube | grep See
1 / 1 [------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 0 p/s
See https://avd.aquasec.com/misconfig/kcv0001
See https://avd.aquasec.com/misconfig/kcv0006
See https://avd.aquasec.com/misconfig/kcv0010
See https://avd.aquasec.com/misconfig/kcv0013
See https://avd.aquasec.com/misconfig/kcv0018
See https://avd.aquasec.com/misconfig/kcv0019
See https://avd.aquasec.com/misconfig/kcv0020
See https://avd.aquasec.com/misconfig/kcv0021
See https://avd.aquasec.com/misconfig/kcv0022
See https://avd.aquasec.com/misconfig/kcv0028
See https://avd.aquasec.com/misconfig/kcv0029

Oct 20 '22 17:10 josedonizetti

Yes , I did run all tests on your branch , but I can check again

Oct 20 '22 17:10 chen-keinan

@josedonizetti looks good rocket , there is something strange experience wise when we run filter by workload we still get rbac results :

I thought this was the idea. I'm treating workload as everything that is not infra, as rbac isn't infra, it returns on the scanning. Should it be different?

Oct 20 '22 17:10 josedonizetti

@josedonizetti looks good rocket , there is something strange experience wise when we run filter by workload we still get rbac results :

I thought this was the idea. I'm treating workload as everything that is not infra, as rbac isn't infra, it returns on the scanning. Should it be different?

Yes ,in a way , but it could be confusing. Anyway it should not hold the pr , we can discuss it later

Oct 20 '22 17:10 chen-keinan

LGTM 🚀 well done !!

Oct 20 '22 17:10 chen-keinan

Follow up issues:

https://github.com/aquasecurity/trivy/issues/3056

Oct 20 '22 18:10 josedonizetti

Just to clarify my understanding, it filters out the misconfiguration results by AVD-ID when the resource is Pod under kube-system so it can know infra components, right?

Exactly!

Oct 23 '22 15:10 josedonizetti

I feel like we may want to show what results each table is displaying. It currently says Summary Report for minikube for all the tables, and I didn't understand how those tables are different at first.

For example,

Summary Report for minikube
===========================

Workload Assessment
[table]

RBAC Assessment
[table]

Infra Assessment
[table]

Oct 23 '22 15:10 knqyf263

Summary Report for minikube

Workload Assessment
┌─────────────┬──────────────────────────────────────┬─────────────────────┬────────────────────┬───────────────────┐
│  Namespace  │               Resource               │   Vulnerabilities   │ Misconfigurations  │      Secrets      │
│             │                                      ├───┬────┬───┬────┬───┼───┬───┬───┬────┬───┼───┬───┬───┬───┬───┤
│             │                                      │ C │ H  │ M │ L  │ U │ C │ H │ M │ L  │ U │ C │ H │ M │ L │ U │
├─────────────┼──────────────────────────────────────┼───┼────┼───┼────┼───┼───┼───┼───┼────┼───┼───┼───┼───┼───┼───┤
│ kube-system │ Service/kube-dns                     │   │    │   │    │   │   │   │ 1 │    │   │   │   │   │   │   │
│ kube-system │ Pod/etcd-minikube                    │   │ 20 │ 4 │    │ 4 │   │ 1 │ 3 │ 7  │   │   │   │   │   │   │
│ kube-system │ Pod/kube-apiserver-minikube          │   │ 2  │ 1 │ 1  │ 2 │   │ 1 │ 3 │ 9  │   │   │   │   │   │   │
│ kube-system │ Pod/kube-scheduler-minikube          │   │ 2  │   │    │   │   │ 1 │ 3 │ 8  │   │   │   │   │   │   │
│ kube-system │ Pod/storage-provisioner              │   │ 9  │ 2 │    │ 3 │   │ 1 │ 5 │ 10 │   │   │   │   │   │   │
│ kube-system │ DaemonSet/kube-proxy                 │   │ 3  │ 2 │ 22 │   │   │ 2 │ 4 │ 10 │   │   │   │   │   │   │
│ kube-system │ Deployment/coredns                   │ 1 │ 3  │ 1 │ 1  │ 4 │   │   │ 3 │ 5  │   │   │   │   │   │   │
│ kube-system │ Pod/kube-controller-manager-minikube │   │ 2  │ 1 │ 1  │ 2 │   │ 1 │ 3 │ 8  │   │   │   │   │   │   │
└─────────────┴──────────────────────────────────────┴───┴────┴───┴────┴───┴───┴───┴───┴────┴───┴───┴───┴───┴───┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN


RBAC Assessment
┌─────────────┬─────────────────────────────────────────────────────┬───────────────────┐
│  Namespace  │                      Resource                       │  RBAC Assessment  │
│             │                                                     ├───┬───┬───┬───┬───┤
│             │                                                     │ C │ H │ M │ L │ U │
├─────────────┼─────────────────────────────────────────────────────┼───┼───┼───┼───┼───┤
│ kube-system │ Role/system:controller:bootstrap-signer             │ 1 │   │   │   │   │
│ kube-system │ Role/system:controller:cloud-provider               │   │   │ 1 │   │   │
│ kube-system │ Role/system:controller:token-cleaner                │ 1 │   │   │   │   │
│ kube-system │ Role/system::leader-locking-kube-scheduler          │   │   │ 1 │   │   │
│ kube-system │ Role/system::leader-locking-kube-controller-manager │   │   │ 1 │   │   │
│ kube-system │ Role/system:persistent-volume-provisioner           │   │ 2 │   │   │   │
└─────────────┴─────────────────────────────────────────────────────┴───┴───┴───┴───┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN


Infra Assessment
┌─────────────┬──────────────────────────────────────┬─────────────────────────────┐
│  Namespace  │               Resource               │ Kubernetes Infra Assessment │
│             │                                      ├────┬────┬────┬─────┬────────┤
│             │                                      │ C  │ H  │ M  │ L   │   U    │
├─────────────┼──────────────────────────────────────┼────┼────┼────┼─────┼────────┤
│ kube-system │ Pod/kube-controller-manager-minikube │    │    │    │ 3   │        │
│ kube-system │ Pod/kube-scheduler-minikube          │    │    │    │ 1   │        │
│ kube-system │ Pod/kube-apiserver-minikube          │    │    │ 1  │ 10  │        │
└─────────────┴──────────────────────────────────────┴────┴────┴────┴─────┴────────┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN

Oct 23 '22 15:10 josedonizetti

@josedonizetti The test is failing now.

Oct 23 '22 17:10 knqyf263

trivy trivy copied to clipboard

feat: add k8s components

Description

Related issues

Checklist

trivy
trivy copied to clipboard