trivy-operator Trivy Operator can't retreive policies behind proxy

What steps did you take and what happened:

The Trivy operator don't use the proxy settings to download the policies.

What did you expect to happen:

Trivy operator been able to retreive the policies

Anything else you would like to add:

I did set the proxy information in the configmap and it's correctly use by the trivy server but not the trivy operator

apiVersion: v1
kind: ConfigMap
metadata:
  name: trivy-operator-trivy-config
  namespace: trivy-system
  labels:
    app.kubernetes.io/instance: trivy-operator
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: trivy-operator
    app.kubernetes.io/version: 0.20.0
    helm.sh/chart: trivy-operator-0.22.0
  annotations:
    meta.helm.sh/release-name: trivy-operator
    meta.helm.sh/release-namespace: trivy-system
data:
  trivy.additionalVulnerabilityReportFields: ''
  trivy.command: fs
  trivy.dbRepository: ghcr.io/aquasecurity/trivy-db
  trivy.dbRepositoryInsecure: 'true'
  trivy.filesystemScanCacheDir: /var/trivyoperator/trivy-db
  trivy.httpProxy: http://my-corp-proxy:3128
  trivy.httpsProxy: http://my-corp-proxy:3128
  trivy.imagePullPolicy: IfNotPresent
  trivy.imageScanCacheDir: /tmp/trivy/.cache
  trivy.includeDevDeps: 'false'
  trivy.insecureRegistry.nexus: my-corp-registry
  trivy.javaDbRepository: ghcr.io/aquasecurity/trivy-java-db
  trivy.mode: Standalone
  trivy.noProxy: 127.0.0.1,.cluster.local
  trivy.repository: my-corp-registry/aquasecurity/trivy
  trivy.resources.limits.cpu: 500m
  trivy.resources.limits.memory: 500M
  trivy.resources.requests.cpu: 100m
  trivy.resources.requests.memory: 100M
  trivy.sbomSources: ''
  trivy.severity: UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL
  trivy.skipJavaDBUpdate: 'false'
  trivy.slow: 'true'
  trivy.supportedConfigAuditKinds: >-
    Workload,Service,Role,ClusterRole,NetworkPolicy,Ingress,LimitRange,ResourceQuota
  trivy.tag: 0.50.1
  trivy.timeout: 5m0s
  trivy.useBuiltinRegoPolicies: 'true'

That make the operator pod generate a lot of error error log:

{"level":"error","ts":"2024-05-14T01:09:10Z","logger":"policyLoader.Get misconfig bundle policies","msg":"failed to load policies","error":"failed to donwload policies: failed to download built-in policies: download error: OCI repository error: 1 error occurred:\n\t* Get \"https://ghcr.io/v2/\": dial tcp: lookup ghcr.io on 10.96.0.10:53: no such host\n\n","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/policy.(*policyLoader).GetPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/loader.go:59\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).loadPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:144\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).Hash\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:114\ngithub.com/aquasecurity/trivy-operator/pkg/configauditreport/controller.(*ResourceController).SetupWithManager.(*ResourceController).reconcileResource.func11\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/configauditreport/controller/resource.go:208\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":"2024-05-14T01:09:10Z","logger":"policyLoader.Get misconfig bundle policies","msg":"failed to load policies","error":"failed to donwload policies: failed to download built-in policies: download error: OCI repository error: 1 error occurred:\n\t* Get \"https://ghcr.io/v2/\": dial tcp: lookup ghcr.io on 10.96.0.10:53: no such host\n\n","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/policy.(*policyLoader).GetPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/loader.go:59\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).loadPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:144\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).Eval\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:199\ngithub.com/aquasecurity/trivy-operator/pkg/configauditreport/controller.evaluate\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/configauditreport/controller/helper.go:45\ngithub.com/aquasecurity/trivy-operator/pkg/configauditreport/controller.(*ResourceController).SetupWithManager.(*ResourceController).reconcileResource.func11\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/configauditreport/controller/resource.go:229\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":"2024-05-14T01:09:10Z","msg":"Reconciler error","controller":"clusterrole","controllerGroup":"rbac.authorization.k8s.io","controllerKind":"ClusterRole","ClusterRole":{"name":"system:kube-dns"},"namespace":"","name":"system:kube-dns","reconcileID":"947aa3bb-e089-48b9-8b3a-eb06858a1c5d","error":"evaluating resource: failed to run policy checks on resources","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":"2024-05-14T01:09:10Z","logger":"policyLoader.Get misconfig bundle policies","msg":"failed to load policies","error":"failed to donwload policies: failed to download built-in policies: download error: OCI repository error: 1 error occurred:\n\t* Get \"https://ghcr.io/v2/\": dial tcp: lookup ghcr.io on 10.96.0.10:53: no such host\n\n","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/policy.(*policyLoader).GetPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/loader.go:59\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).loadPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:144\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).Hash\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:114\ngithub.com/aquasecurity/trivy-operator/pkg/configauditreport/controller.(*ResourceController).SetupWithManager.(*ResourceController).reconcileResource.func11\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/configauditreport/controller/resource.go:208\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}

Environment:

Trivy-Operator version (use trivy-operator version): 0.20.0
Kubernetes version (use kubectl version): v1.28.2
OS (macOS 10.15, Windows 10, Ubuntu 19.10 etc): AlmaLinux 9

May 14 '24 01:05 urcus

I think this is a proxy server issue.

According to the error log you attached, you can see that the DNS Resolve process is resolving to 10.96.0.10, which is not the public IP address of ghcr.io.

May 15 '24 00:05 bunseokbot

@bunseokbot thanks for your answer

The issue is that the request is not send to the proxy as the 10.96.0.10 is the IP of the CoreDNS service, and i don't find any presence of the proxy configuration inside the Operator pod.

May 15 '24 00:05 urcus

@urcus I checked and it seems that trivy-operator can't set http/https proxy while downloading policy from ghcr.io. @chen-keinan if you don't mind, can I add the feature?

https://github.com/aquasecurity/trivy-operator/blob/015e5140454dac77b7eb4c3adb024315974f9d24/pkg/operator/operator.go#L427

May 15 '24 05:05 bunseokbot

@bunseokbot sure, go for it

May 15 '24 05:05 chen-keinan