trivy-operator icon indicating copy to clipboard operation
trivy-operator copied to clipboard

Published 20 hours ago • verified publisher icon aquasecurity

Unable to initialize a scanner: {docker,containerd,podman,remote} error?

Open sheeeng opened this issue 1 year ago • 20 comments

What steps did you take and what happened:

{"level":"error","ts":"2024-05-21T10:46:12Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy/scan-vulnerabilityreport-6543cd987d","container":"linkerd-proxy","status.reason":"Error","status.message":"2024-05-21T10:46:09.102Z\t\u001b[31mFATAL\u001b[0m\timage scan error: scan error: unable to initialize a scanner: unable to initialize a remote image scanner: 4 errors occurred:\n\t* docker error: unable to inspect the image (cr.l5d.io/linkerd/proxy:stable-2.14.10): Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?\n\t* containerd error: containerd socket not found: /run/containerd/containerd.sock\n\t* podman error: unable to initialize Podman client: no podman socket found: stat podman/podman.sock: no such file or directory\n\t* remote error: Get \"https://cr.l5d.io/v2/\": dial tcp 3.67.33.93:443: connect: connection refused\n\n\n","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}

What did you expect to happen:

The error message should not appear.

Anything else you would like to add:

No.

Environment:

  • Trivy-Operator version (use trivy-operator version): 0.22.1
  • Kubernetes Server version (use kubectl version): v1.28.5
  • OS (macOS 10.15, Windows 10, Ubuntu 19.10 etc): macOS 14.5 (23F79)

sheeeng avatar May 21 '24 11:05 sheeeng

@sheeeng is this image cr.l5d.io/linkerd/proxy:stable-2.14.10 pulled from private registry? if so how are you setting secrets ?

chen-keinan avatar May 21 '24 11:05 chen-keinan

@sheeeng is this image cr.l5d.io/linkerd/proxy:stable-2.14.10 pulled from private registry? if so how are you setting secrets ?

Thank you for the tip. I will check on it. Get back to you as soon as I have more information.

sheeeng avatar May 21 '24 12:05 sheeeng

@sheeeng checkout our private registry docs

chen-keinan avatar May 21 '24 12:05 chen-keinan 

proxy:
  image:
    name: ghcr.io/linkerd/proxy

I have tried to change the default container image name to use ghcr.io prefix, but to no avail.

{"level":"error","ts":"2024-05-22T08:06:58Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy/scan-vulnerabilityreport-564746fb59","container":"linkerd-proxy","status.reason":"Error","status.message":"2024-05-22T08:06:55.383Z\t\u001b[31mFATAL\u001b[0m\timage scan error: scan error: unable to initialize a scanner: unable to initialize a remote image scanner: 4 errors occurred:\n\t* docker error: unable to inspect the image (ghcr.io/linkerd/proxy:stable-2.14.10): Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?\n\t* containerd error: containerd socket not found: /run/containerd/containerd.sock\n\t* podman error: unable to initialize Podman client: no podman socket found: stat podman/podman.sock: no such file or directory\n\t* remote error: Get \"https://ghcr.io/v2/\": dial tcp 140.82.121.34:443: connect: connection refused\n\n\n","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}

I guess trivy-operator needs personal access token to access ghcr.io?

sheeeng avatar May 22 '24 08:05 sheeeng 

proxy:
  image:
    name: ghcr.io/linkerd/proxy

I have tried to change the default container image name to use ghcr.io prefix, but to no avail.

{"level":"error","ts":"2024-05-22T08:06:58Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy/scan-vulnerabilityreport-564746fb59","container":"linkerd-proxy","status.reason":"Error","status.message":"2024-05-22T08:06:55.383Z\t\u001b[31mFATAL\u001b[0m\timage scan error: scan error: unable to initialize a scanner: unable to initialize a remote image scanner: 4 errors occurred:\n\t* docker error: unable to inspect the image (ghcr.io/linkerd/proxy:stable-2.14.10): Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?\n\t* containerd error: containerd socket not found: /run/containerd/containerd.sock\n\t* podman error: unable to initialize Podman client: no podman socket found: stat podman/podman.sock: no such file or directory\n\t* remote error: Get \"https://ghcr.io/v2/\": dial tcp 140.82.121.34:443: connect: connection refused\n\n\n","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}

I guess trivy-operator needs personal access token to access ghcr.io?

@sheeeng if it is private then yes if public no

chen-keinan avatar May 22 '24 08:05 chen-keinan

@sheeeng if it is private then yes if public no

According to Using A Private Docker Repository | Linkerd, all of the Linkerd images are publicly available at the GitHub Container Registry.

Hmm. I've changed the default container image name to use ghcr.io prefix, but to no avail. Are there other debugging steps that I can try out?

sheeeng avatar May 23 '24 06:05 sheeeng

@sheeeng it works for other public images but for this one it fails ?

chen-keinan avatar May 23 '24 06:05 chen-keinan

@sheeeng it works for other public images but for this one it fails ?

Yes, it works for other public images.

$ kubectl get vulnerabilityreports.aquasecurity.github.io --all-namespaces --output json | jq '.items[] | select(.kind=="VulnerabilityReport")' | jq --raw-output '[.report.updateTimestamp, .report.registry.server, .report.artifact.repository, .report.artifact.tag, ("Vulnerabilities:" + (.report.vulnerabilities | length | tostring)), ("CriticalVulnerabilities:" + (.report.summary.criticalCount | tostring))] | @csv' | sort | grep aquasecurity
"2024-05-22T13:11:36Z","ghcr.io","aquasecurity/trivy","0.50.2","Vulnerabilities:16","CriticalVulnerabilities:1"
"2024-05-22T13:15:25Z","ghcr.io","aquasecurity/trivy-operator","0.20.1","Vulnerabilities:12","CriticalVulnerabilities:0"

sheeeng avatar May 23 '24 07:05 sheeeng

@sheeeng it works for other public images but for this one it fails ?

Yes, it works for other public images.

$ kubectl get vulnerabilityreports.aquasecurity.github.io --all-namespaces --output json | jq '.items[] | select(.kind=="VulnerabilityReport")' | jq --raw-output '[.report.updateTimestamp, .report.registry.server, .report.artifact.repository, .report.artifact.tag, ("Vulnerabilities:" + (.report.vulnerabilities | length | tostring)), ("CriticalVulnerabilities:" + (.report.summary.criticalCount | tostring))] | @csv' | sort | grep aquasecurity
"2024-05-22T13:11:36Z","ghcr.io","aquasecurity/trivy","0.50.2","Vulnerabilities:16","CriticalVulnerabilities:1"
"2024-05-22T13:15:25Z","ghcr.io","aquasecurity/trivy-operator","0.20.1","Vulnerabilities:12","CriticalVulnerabilities:0"

I think the issue relate to settings, you mention that you tried to change the image name from cr.l5d.io/linkerd/proxy:stable-2.14.10 to ghcr.io/linkerd/proxy something in the way you changed it, as I was able to download the image directly (its public)

chen-keinan avatar May 23 '24 08:05 chen-keinan

I think the issue relate to settings, you mention that you tried to change the image name from cr.l5d.io/linkerd/proxy:stable-2.14.10 to ghcr.io/linkerd/proxy something in the way you changed it, as I was able to download the image directly (its public)

Thank you for trying it on your side. I will further debug on my side.

sheeeng avatar May 23 '24 08:05 sheeeng

I was able to pull container images from both cr.l5d.io and ghcr.io without authentication successfully.

podman pull cr.l5d.io/linkerd/controller:stable-2.14.10
podman pull ghcr.io/linkerd/proxy:stable-2.14.10

It can generate VulnerabilityReport inside a minimal kind cluster using Podman Desktop.

$ kubectl get vulnerabilityreports.aquasecurity.github.io --all-namespaces --output json | jq '.items[] | select(.kind=="VulnerabilityReport")' | jq --raw-output '[.report.updateTimestamp, .report.registry.server, .report.artifact.repository, .report.artifact.tag, ("Vulnerabilities:" + (.report.vulnerabilities | length | tostring)), ("CriticalVulnerabilities:" + (.report.summary.criticalCount | tostring))] | @csv' | sort
"2024-05-24T07:43:21Z","index.docker.io","kindest/kindnetd","v20240202-8f1494ea","Vulnerabilities:77","CriticalVulnerabilities:1"
"2024-05-24T07:43:21Z","registry.k8s.io","etcd","3.5.12-0","Vulnerabilities:38","CriticalVulnerabilities:0"
"2024-05-24T07:43:22Z","ghcr.io","projectcontour/contour","v1.24.2","Vulnerabilities:31","CriticalVulnerabilities:2"
"2024-05-24T07:43:22Z","registry.k8s.io","kube-controller-manager","v1.30.0","Vulnerabilities:3","CriticalVulnerabilities:0"
"2024-05-24T07:43:23Z","registry.k8s.io","kube-proxy","v1.30.0","Vulnerabilities:30","CriticalVulnerabilities:0"
"2024-05-24T07:43:26Z","ghcr.io","aquasecurity/trivy-operator","0.21.1","Vulnerabilities:1","CriticalVulnerabilities:0"
"2024-05-24T07:43:26Z","registry.k8s.io","kube-apiserver","v1.30.0","Vulnerabilities:3","CriticalVulnerabilities:0"
"2024-05-24T07:43:28Z","registry.k8s.io","coredns/coredns","v1.11.1","Vulnerabilities:21","CriticalVulnerabilities:0"
"2024-05-24T07:43:35Z","ghcr.io","projectcontour/contour","v1.24.2","Vulnerabilities:31","CriticalVulnerabilities:2"
"2024-05-24T07:43:35Z","ghcr.io","projectcontour/contour","v1.24.2","Vulnerabilities:31","CriticalVulnerabilities:2"
"2024-05-24T07:43:35Z","index.docker.io","envoyproxy/envoy","v1.25.2","Vulnerabilities:89","CriticalVulnerabilities:0"
"2024-05-24T07:43:52Z","index.docker.io","kindest/local-path-provisioner","v20240202-8f1494ea","Vulnerabilities:38","CriticalVulnerabilities:0"
"2024-05-24T07:43:58Z","ghcr.io","projectcontour/contour","v1.24.2","Vulnerabilities:31","CriticalVulnerabilities:2"
"2024-05-24T07:43:58Z","registry.k8s.io","kube-scheduler","v1.30.0","Vulnerabilities:3","CriticalVulnerabilities:0"
"2024-05-24T07:44:10Z","index.docker.io","library/nginx","1.16","Vulnerabilities:471","CriticalVulnerabilities:40"
"2024-05-24T08:29:12Z","cr.l5d.io","linkerd/controller","stable-2.14.10","Vulnerabilities:19","CriticalVulnerabilities:0"
"2024-05-24T08:29:12Z","cr.l5d.io","linkerd/controller","stable-2.14.10","Vulnerabilities:19","CriticalVulnerabilities:0"
"2024-05-24T08:29:12Z","cr.l5d.io","linkerd/controller","stable-2.14.10","Vulnerabilities:19","CriticalVulnerabilities:0"
"2024-05-24T08:29:12Z","cr.l5d.io","linkerd/controller","stable-2.14.10","Vulnerabilities:19","CriticalVulnerabilities:0"
"2024-05-24T08:29:12Z","cr.l5d.io","linkerd/controller","stable-2.14.10","Vulnerabilities:19","CriticalVulnerabilities:0"
"2024-05-24T08:29:12Z","cr.l5d.io","linkerd/policy-controller","stable-2.14.10","Vulnerabilities:0","CriticalVulnerabilities:0"
"2024-05-24T08:29:12Z","cr.l5d.io","linkerd/proxy","stable-2.14.10","Vulnerabilities:40","CriticalVulnerabilities:0"
"2024-05-24T08:29:12Z","cr.l5d.io","linkerd/proxy","stable-2.14.10","Vulnerabilities:40","CriticalVulnerabilities:0"
"2024-05-24T08:29:12Z","cr.l5d.io","linkerd/proxy","stable-2.14.10","Vulnerabilities:40","CriticalVulnerabilities:0"
"2024-05-24T08:29:12Z","cr.l5d.io","linkerd/proxy-init","v2.2.3","Vulnerabilities:27","CriticalVulnerabilities:0"
"2024-05-24T08:29:12Z","cr.l5d.io","linkerd/proxy-init","v2.2.3","Vulnerabilities:27","CriticalVulnerabilities:0"
"2024-05-24T08:29:12Z","cr.l5d.io","linkerd/proxy-init","v2.2.3","Vulnerabilities:27","CriticalVulnerabilities:0"
"2024-05-24T08:30:34Z","docker.l5d.io","buoyantio/emojivoto-web","v11","Vulnerabilities:990","CriticalVulnerabilities:61"
"2024-05-24T08:32:56Z","docker.l5d.io","buoyantio/emojivoto-voting-svc","v11","Vulnerabilities:904","CriticalVulnerabilities:58"
"2024-05-24T08:32:58Z","docker.l5d.io","buoyantio/emojivoto-web","v11","Vulnerabilities:990","CriticalVulnerabilities:61"
"2024-05-24T08:33:28Z","docker.l5d.io","buoyantio/emojivoto-emoji-svc","v11","Vulnerabilities:904","CriticalVulnerabilities:58"
"2024-05-24T08:33:28Z","docker.l5d.io","buoyantio/emojivoto-web","v11","Vulnerabilities:990","CriticalVulnerabilities:61"
"2024-05-24T08:35:16Z","index.docker.io","prom/prometheus","v2.48.0","Vulnerabilities:24","CriticalVulnerabilities:0"
"2024-05-24T08:35:17Z","cr.l5d.io","linkerd/metrics-api","stable-2.14.10","Vulnerabilities:19","CriticalVulnerabilities:0"
"2024-05-24T08:35:17Z","cr.l5d.io","linkerd/tap","stable-2.14.10","Vulnerabilities:19","CriticalVulnerabilities:0"
"2024-05-24T08:35:17Z","cr.l5d.io","linkerd/tap","stable-2.14.10","Vulnerabilities:19","CriticalVulnerabilities:0"
"2024-05-24T08:35:17Z","cr.l5d.io","linkerd/web","stable-2.14.10","Vulnerabilities:19","CriticalVulnerabilities:0"

sheeeng avatar May 24 '24 09:05 sheeeng

I managed to reproduce this issue using Linkerd with CNI plugin on local Podman Desktop using following steps.

The previous experiment that produces no error was using plain Linkerd only.

  • Create a kind cluster in Podman Desktop.

Generating your own mTLS root certificates

$ step certificate create root.linkerd.cluster.local ca.crt ca.key \
--profile root-ca --no-password --insecure
Your certificate has been saved in ca.crt.
Your private key has been saved in ca.key.
$ step certificate create identity.linkerd.cluster.local issuer.crt issuer.key \
--profile intermediate-ca --not-after 8760h --no-password --insecure \
--ca ca.crt --ca-key ca.key
Your certificate has been saved in issuer.crt.
Your private key has been saved in issuer.key.
$

Install Linkerd CNI with Helm

$ helm repo add linkerd https://helm.linkerd.io/stable
"linkerd" has been added to your repositories
$
$ helm install linkerd-cni -n linkerd-cni --create-namespace linkerd/linkerd2-cni
NAME: linkerd-cni
...
TEST SUITE: None
$
$ linkerd check --pre --linkerd-cni-enabled
...
Status check results are √
$
$ helm install linkerd-crds linkerd/linkerd-crds \
  --set cniEnabled=true \
  -n linkerd --create-namespace
NAME: linkerd-crds
...
Looking for more? Visit https://linkerd.io/2/getting-started/
$
$ helm install linkerd-control-plane \
  --set cniEnabled=true \
  -n linkerd \
  --set-file identityTrustAnchorsPEM=ca.crt \
  --set-file identity.issuer.tls.crtPEM=issuer.crt \
  --set-file identity.issuer.tls.keyPEM=issuer.key \
  linkerd/linkerd-control-plane
NAME: linkerd-control-plane
...
Looking for more? Visit https://linkerd.io/2/getting-started/
$

Install Trivy Operator with Helm

$  helm repo add aqua https://aquasecurity.github.io/helm-charts/
"aqua" has been added to your repositories
$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "aqua" chart repository
...Successfully got an update from the "linkerd" chart repository
Update Complete. ⎈Happy Helming!⎈
$
$ cat trivy-operator-values.yaml
operator:
  builtInTrivyServer: true
trivy:
  mode: "ClientServer"
$
$ helm install trivy-operator aqua/trivy-operator \
     --namespace trivy-system \
     --create-namespace \
     --values trivy-operator-values.yaml
NAME: trivy-operator
...
    kubectl logs -n trivy-system deployment/trivy-operator
$

$ helm list --all --all-namespaces
NAME                 	NAMESPACE   	REVISION	UPDATED                              	STATUS  	CHART                        	APP VERSION
linkerd-cni          	linkerd-cni 	1       	2024-06-17 10:43:53.950234 +0200 CEST	deployed	linkerd2-cni-30.12.2         	stable-2.14.10
linkerd-control-plane	linkerd     	1       	2024-06-17 10:44:20.04936 +0200 CEST 	deployed	linkerd-control-plane-1.16.11	stable-2.14.10
linkerd-crds         	linkerd     	1       	2024-06-17 10:44:11.914834 +0200 CEST	deployed	linkerd-crds-1.8.0
trivy-operator       	trivy-system	1       	2024-06-17 10:49:07.889866 +0200 CEST	deployed	trivy-operator-0.23.3        	0.21.3

Get Error Logs

$ kubectl logs --selector "app.kubernetes.io/name=trivy-operator" --all-containers --namespace trivy-system --context kind-kind-cluster --max-log-requests=6 --follow
{"level":"error","ts":"2024-05-27T09:34:40Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-58dbf59895","container":"linkerd-network-validator","status.reason":"Error","status.message":"","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2024-05-27T09:34:40Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-58dbf59895","container":"linkerd-proxy","status.reason":"Error","status.message":"","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2024-05-27T09:34:40Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-58dbf59895","container":"policy","status.reason":"Error","status.message":"","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2024-05-27T09:34:40Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-58dbf59895","container":"sp-validator","status.reason":"Error","status.message":"","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2024-05-27T09:34:41Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-9fdb684cc","container":"identity","status.reason":"Error","status.message":"","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2024-05-27T09:34:41Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-9fdb684cc","container":"linkerd-network-validator","status.reason":"Error","status.message":"","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2024-05-27T09:34:41Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-9fdb684cc","container":"linkerd-proxy","status.reason":"Error","status.message":"","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2024-05-27T09:34:44Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-d68d89976","container":"linkerd-network-validator","status.reason":"Error","status.message":"","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2024-05-27T09:34:44Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-d68d89976","container":"linkerd-proxy","status.reason":"Error","status.message":"","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2024-05-27T09:34:44Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-d68d89976","container":"proxy-injector","status.reason":"Error","status.message":"","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}

Stacktrace

github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers
        /home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353
github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1
        /home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80
sigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile
        /home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
        /home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
        /home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
        /home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
        /home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222

sheeeng avatar May 27 '24 09:05 sheeeng

Same stacktrace is found on https://github.com/aquasecurity/trivy-operator/issues/2101#issue-2317860263 too.

sheeeng avatar May 27 '24 10:05 sheeeng

@sheeeng what is the status of this issue have you found workaround

chen-keinan avatar Jun 23 '24 11:06 chen-keinan

@sheeeng the errors issue fixed by trivy-operator v0.21.3 feel free to re-open if you think issue still exist

chen-keinan avatar Jun 24 '24 08:06 chen-keinan

@sheeeng what is the status of this issue have you found workaround @sheeeng the errors issue fixed by trivy-operator v0.21.3 feel free to re-open if you think issue still exist

@chen-keinan, The issue is still reproducible with trivy-operator v0.21.3. See https://github.com/aquasecurity/trivy-operator/issues/2095#issuecomment-2133093233 minimal example.

sheeeng avatar Jun 24 '24 08:06 sheeeng

@sheeeng can you clarify which errors, you mean the log errors ?

chen-keinan avatar Jun 24 '24 08:06 chen-keinan

@sheeeng can you clarify which errors, you mean the log errors ?

@chen-keinan, these error logs:

$ kubectl logs --selector "app.kubernetes.io/name=trivy-operator" --all-containers --namespace trivy-system --context kind-kind-cluster --max-log-requests=6 --follow
{"level":"error","ts":"2024-05-27T09:34:40Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-58dbf59895","container":"linkerd-network-validator","status.reason":"Error","status.message":"","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2024-05-27T09:34:40Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-58dbf59895","container":"linkerd-proxy","status.reason":"Error","status.message":"","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2024-05-27T09:34:40Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-58dbf59895","container":"policy","status.reason":"Error","status.message":"","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2024-05-27T09:34:40Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-58dbf59895","container":"sp-validator","status.reason":"Error","status.message":"","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2024-05-27T09:34:41Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-9fdb684cc","container":"identity","status.reason":"Error","status.message":"","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2024-05-27T09:34:41Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-9fdb684cc","container":"linkerd-network-validator","status.reason":"Error","status.message":"","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2024-05-27T09:34:41Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-9fdb684cc","container":"linkerd-proxy","status.reason":"Error","status.message":"","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2024-05-27T09:34:44Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-d68d89976","container":"linkerd-network-validator","status.reason":"Error","status.message":"","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2024-05-27T09:34:44Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-d68d89976","container":"linkerd-proxy","status.reason":"Error","status.message":"","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2024-05-27T09:34:44Z","logger":"reconciler.scan job","msg":"Scan job container","job":"trivy-system/scan-vulnerabilityreport-d68d89976","container":"proxy-injector","status.reason":"Error","status.message":"","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).completedContainers\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:353\ngithub.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:80\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}

sheeeng avatar Jun 24 '24 08:06 sheeeng

the errors appear on linkerd related resources only ?

chen-keinan avatar Jun 24 '24 08:06 chen-keinan

the errors appear on linkerd related resources only ?

Yes, the errors appear with the given minimal example.

sheeeng avatar Jun 24 '24 10:06 sheeeng

This issue is stale because it has been labeled with inactivity.

github-actions[bot] avatar Aug 24 '24 00:08 github-actions[bot]