trivy icon indicating copy to clipboard operation
trivy copied to clipboard
Published 2 months ago • verified publisher icon aquasecurity

failed to download vulnerability DB using the docker image

Open kforner opened this issue 3 years ago • 16 comments

Description

From one server, the DB can not be downloaded using docker:

$ docker run --rm --privileged -v /var/run/docker.sock:/var/run/docker.sock  -i aquasec/trivy:latest image  --severity "MEDIUM,HIGH,CRITICAL" --no-progress  --debug ubuntu:22.10
2022-09-01T13:22:12.806Z	DEBUG	Severities: ["MEDIUM" "HIGH" "CRITICAL"]
2022-09-01T13:22:12.810Z	DEBUG	cache dir:  /root/.cache/trivy
2022-09-01T13:22:12.810Z	DEBUG	There is no valid metadata file: unable to open a file: open /root/.cache/trivy/db/metadata.json: no such file or directory
2022-09-01T13:22:12.811Z	INFO	Need to update DB
2022-09-01T13:22:12.811Z	INFO	DB Repository: ghcr.io/aquasecurity/trivy-db
2022-09-01T13:22:12.811Z	INFO	Downloading DB...
2022-09-01T13:22:12.811Z	DEBUG	no metadata file
2022-09-01T13:22:41.910Z	FATAL	init error:
    github.com/aquasecurity/trivy/pkg/commands/artifact.Run
        /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:362
  - DB error:
    github.com/aquasecurity/trivy/pkg/commands/artifact.NewRunner
        /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:121
  - failed to download vulnerability DB:
    github.com/aquasecurity/trivy/pkg/commands/operation.DownloadDB
        /home/runner/work/trivy/trivy/pkg/commands/operation/operation.go:117
  - OCI artifact error:
    github.com/aquasecurity/trivy/pkg/db.(*Client).Download
        /home/runner/work/trivy/trivy/pkg/db/db.go:154
  - OCI artifact error:
    github.com/aquasecurity/trivy/pkg/db.(*Client).initOCIArtifact
        /home/runner/work/trivy/trivy/pkg/db/db.go:194
  - OCI repository error:
    github.com/aquasecurity/trivy/pkg/oci.NewArtifact
        /home/runner/work/trivy/trivy/pkg/oci/artifact.go:69
  - Get "https://ghcr.io/v2/": dial tcp: lookup ghcr.io: i/o timeout

The exact same command works on my personal computer. They both use the same ubuntu and docker versions.

On the server, I can work-around it using --insecure:

$ docker run --rm --privileged -v /var/run/docker.sock:/var/run/docker.sock  -i aquasec/trivy:latest image  --severity "MEDIUM,HIGH,CRITICAL" --no-progress --insecure --debug ubuntu:22.10
2022-09-01T14:03:33.060Z	DEBUG	Severities: ["MEDIUM" "HIGH" "CRITICAL"]
2022-09-01T14:03:33.063Z	DEBUG	cache dir:  /root/.cache/trivy
2022-09-01T14:03:33.063Z	DEBUG	There is no valid metadata file: unable to open a file: open /root/.cache/trivy/db/metadata.json: no such file or directory
2022-09-01T14:03:33.063Z	INFO	Need to update DB
2022-09-01T14:03:33.063Z	INFO	DB Repository: ghcr.io/aquasecurity/trivy-db
2022-09-01T14:03:33.063Z	INFO	Downloading DB...
2022-09-01T14:03:33.063Z	DEBUG	no metadata file
2022-09-01T14:03:49.233Z	DEBUG	Updating database metadata...
2022-09-01T14:03:49.234Z	DEBUG	DB Schema: 2, UpdatedAt: 2022-09-01 12:06:40.991108038 +0000 UTC, NextUpdate: 2022-09-01 18:06:40.991107538 +0000 UTC, DownloadedAt: 2022-09-01 14:03:49.23375211 +0000 UTC
2022-09-01T14:03:49.234Z	INFO	Vulnerability scanning is enabled
2022-09-01T14:03:49.234Z	DEBUG	Vulnerability type:  [os library]
2022-09-01T14:03:49.234Z	INFO	Secret scanning is enabled
2022-09-01T14:03:49.234Z	INFO	If your scanning is slow, please try '--security-checks vuln' to disable secret scanning
2022-09-01T14:03:49.234Z	INFO	Please see also https://aquasecurity.github.io/trivy/v0.31.3/docs/secret/scanning/#recommendation for faster secret detection
2022-09-01T14:03:49.241Z	DEBUG	No secret config detected: trivy-secret.yaml
2022-09-01T14:03:49.242Z	DEBUG	Image ID: sha256:15a38249db7a639fe4781bc597b57ec2c936e5b576eb54f2f281658318d62613
2022-09-01T14:03:49.242Z	DEBUG	Diff IDs: [sha256:6f48304a39b941322886ad7cfddcb7a8689b26c05d13e814332e372fd03c6f1b]
2022-09-01T14:03:49.242Z	DEBUG	Base Layers: []
2022-09-01T14:03:49.242Z	DEBUG	Missing image ID in cache: sha256:15a38249db7a639fe4781bc597b57ec2c936e5b576eb54f2f281658318d62613
2022-09-01T14:03:49.242Z	DEBUG	Missing diff ID in cache: sha256:6f48304a39b941322886ad7cfddcb7a8689b26c05d13e814332e372fd03c6f1b
2022-09-01T14:03:50.940Z	DEBUG	Loading the the default license classifier...
2022-09-01T14:03:53.098Z	INFO	Detected OS: ubuntu
2022-09-01T14:03:53.098Z	WARN	This OS version is not on the EOL list: ubuntu 22.10
2022-09-01T14:03:53.098Z	INFO	Detecting Ubuntu vulnerabilities...
2022-09-01T14:03:53.098Z	DEBUG	ubuntu: os version: 22.10
2022-09-01T14:03:53.098Z	DEBUG	ubuntu: the number of packages: 94
2022-09-01T14:03:53.098Z	INFO	Number of language-specific files: 0
2022-09-01T14:03:53.098Z	WARN	This OS version is no longer supported by the distribution: ubuntu 22.10
2022-09-01T14:03:53.098Z	WARN	The vulnerability detection may be insufficient because security updates are not provided

ubuntu:22.10 (ubuntu 22.10)
===========================
Total: 0 (MEDIUM: 0, HIGH: 0, CRITICAL: 0)

This is very similar to issue https://github.com/aquasecurity/trivy/discussions/2689#discussioncomment-3407220 I tried docker logout ghcr.io with no success.

I upgraded this server to ubuntu 20.04 and the latest docker version 20.10.17 today, so this is probably related.

kforner avatar Sep 01 '22 14:09 kforner

I guess your server is under a proxy or something similar. Please make sure your server is able to reach ghcr.io.

knqyf263 avatar Sep 01 '22 14:09 knqyf263

it is not under a proxy, and it works with --insecure so it can reach ghcr.io.

kforner avatar Sep 01 '22 14:09 kforner

and thank you @knqyf263 for your help.

kforner avatar Sep 01 '22 15:09 kforner

No, I meant without --insecure.

if it shows below, it would be fine.

$ curl https://ghcr.io/v2/
{"errors":[{"code":"UNAUTHORIZED","message":"authentication required"}]}

knqyf263 avatar Sep 01 '22 15:09 knqyf263 

$ curl https://ghcr.io/v2/
{"errors":[{"code":"UNAUTHORIZED","message":"authentication required"}]}

and it still does not work without --insecure

kforner avatar Sep 01 '22 15:09 kforner

This issue is stale because it has been labeled with inactivity.

github-actions[bot] avatar Nov 01 '22 00:11 github-actions[bot]

I'm having the same errors with trivy 0.34.0

$ docker run --rm aquasec/trivy --version
Version: 0.34.0

$ docker run --rm aquasec/trivy --debug --insecure server --listen localhost:8080
2022-11-16T12:46:06.166Z        DEBUG   cache dir:  /root/.cache/trivy
2022-11-16T12:46:06.166Z        DEBUG   There is no valid metadata file: unable to open a file: open /root/.cache/trivy/db/metadata.json: no such file or directory
2022-11-16T12:46:06.166Z        INFO    Need to update DB
2022-11-16T12:46:06.166Z        INFO    DB Repository: ghcr.io/aquasecurity/trivy-db
2022-11-16T12:46:06.166Z        INFO    Downloading DB...
2022-11-16T12:46:06.166Z        DEBUG   no metadata file
2022-11-16T12:46:06.176Z        FATAL   failed to download vulnerability DB:
    github.com/aquasecurity/trivy/pkg/commands/operation.DownloadDB
        /home/runner/work/trivy/trivy/pkg/commands/operation/operation.go:117
  - OCI artifact error:
    github.com/aquasecurity/trivy/pkg/db.(*Client).Download
        /home/runner/work/trivy/trivy/pkg/db/db.go:154
  - OCI artifact error:
    github.com/aquasecurity/trivy/pkg/db.(*Client).initOCIArtifact
        /home/runner/work/trivy/trivy/pkg/db/db.go:194
  - OCI repository error:
    github.com/aquasecurity/trivy/pkg/oci.NewArtifact
        /home/runner/work/trivy/trivy/pkg/oci/artifact.go:69
  - Get "https://ghcr.io/v2/": dial tcp 140.82.121.33:443: connect: connection refused

danielnbalasoiu avatar Nov 16 '22 12:11 danielnbalasoiu 

github.com/aquasecurity/trivy/pkg/oci.NewArtifact
    /home/runner/work/trivy/trivy/pkg/oci/artifact.go:69
  • Get "https://ghcr.io/v2/": dial tcp 140.82.121.33:443: connect: connection refused

@danielnbalasoiu thanks for your report! we're trying to clarify this error right now

could give us a bit more information?

what is your OS/arch? ex: MasOS/arm64, Linux etc/

Do you try to run Trivy with --insecure flag?

thanks a lot

afdesk avatar Nov 23 '22 11:11 afdesk

I'm running it on a Linux (Ubuntu) x86_64 arch. From the connectivity stand point, all outgoing traffic is blocked unless HTTP(S)_PPROXY enviroment variables are set. The proxy server has it's own certificate

What have I tried?

  • I overrode trivy container entrypoint and manually configured proxy env vars
  • I defined the proxy env vars in a .docker-env file and run the container
docker run --env-file .docker-env --net=host --rm aquasec/trivy --debug --insecure server --listen localhost:8080
  • I run the same command as above and include --debug & --insecure flags

Debug info

debug1 
$ docker run --env-file .docker-env --net=host -it --rm --entrypoint sh aquasec/trivy
Unable to find image 'aquasec/trivy:latest' locally
latest: Pulling from aquasec/trivy
213ec9aee27d: Pull complete
ad53b2e0219a: Pull complete
2399349afd31: Pull complete
dc0298aa2f10: Pull complete
Digest: sha256:a5544f44ca957135921410f4d3fa340d42b6ab56bbb6bf7406d783df9e84f95f
Status: Downloaded newer image for aquasec/trivy:latest

/ # sed -i 's/https/http/g' /etc/apk/repositories
/ # apk update
fetch http://dl-cdn.alpinelinux.org/alpine/v3.16/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.16/community/x86_64/APKINDEX.tar.gz
v3.16.3-13-g4d933a1fa3 [http://dl-cdn.alpinelinux.org/alpine/v3.16/main]
v3.16.3-12-g2affb64843 [http://dl-cdn.alpinelinux.org/alpine/v3.16/community]
OK: 17041 distinct packages available

/ # trivy server --listen localhost:8080
2022-11-23T16:32:59.674Z        INFO    Need to update DB
2022-11-23T16:32:59.674Z        INFO    DB Repository: ghcr.io/aquasecurity/trivy-db
2022-11-23T16:32:59.674Z        INFO    Downloading DB...
2022-11-23T16:32:59.720Z        FATAL   failed to download vulnerability DB: OCI artifact error: OCI artifact error: OCI repository error: Get "https://ghcr.io/v2/": dial tcp 140.82.121.33:443: connect: connection refused
debug2 
$ docker run --env-file .docker-env --net=host --rm aquasec/trivy --debug --insecure server --listen localhost:8080
2022-11-23T16:39:41.181Z        DEBUG   cache dir:  /home/runner/.cache/trivy
2022-11-23T16:39:41.182Z        DEBUG   There is no valid metadata file: unable to open a file: open /home/runner/.cache/trivy/db/metadata.json: no such file or directory
2022-11-23T16:39:41.182Z        INFO    Need to update DB
2022-11-23T16:39:41.182Z        INFO    DB Repository: ghcr.io/aquasecurity/trivy-db
2022-11-23T16:39:41.182Z        INFO    Downloading DB...
2022-11-23T16:39:41.182Z        DEBUG   no metadata file
2022-11-23T16:39:41.207Z        FATAL   failed to download vulnerability DB:
  github.com/aquasecurity/trivy/pkg/commands/operation.DownloadDB
      /home/runner/work/trivy/trivy/pkg/commands/operation/operation.go:117
- OCI artifact error:
  github.com/aquasecurity/trivy/pkg/db.(*Client).Download
      /home/runner/work/trivy/trivy/pkg/db/db.go:154
- OCI artifact error:
  github.com/aquasecurity/trivy/pkg/db.(*Client).initOCIArtifact
      /home/runner/work/trivy/trivy/pkg/db/db.go:194
- OCI repository error:
  github.com/aquasecurity/trivy/pkg/oci.NewArtifact
      /home/runner/work/trivy/trivy/pkg/oci/artifact.go:69
- Get "https://ghcr.io/v2/": dial tcp 140.82.121.34:443: connect: connection refused

danielnbalasoiu avatar Nov 23 '22 16:11 danielnbalasoiu

@danielnbalasoiu thanks for your details. will investigate more

afdesk avatar Nov 23 '22 17:11 afdesk

@danielnbalasoiu if you use proxy, --insecure flag doesn't work for you. it's a known issue.

I can reproduce connect: connection refused, for example when Trivy tries to download the db from incorrect sources. so it seems there is some problem with routing on your host/proxy.

Could you check it?

afdesk avatar Nov 25 '22 14:11 afdesk

note: 140.82.121.34:443 is a correct IP for ghcr.io. if i set it in my local hosts file, Trivy will download the db

afdesk avatar Nov 25 '22 14:11 afdesk

@afdesk , I tested this scenario:

$ docker run --add-host ghcr.io:140.82.121.34 --env-file .docker-env --net=host --rm --entrypoint=cat aquasec/trivy /etc/hosts
127.0.0.1       localhost
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
140.82.121.34   ghcr.io

$ docker run --add-host ghcr.io:140.82.121.34 --env-file .docker-env --net=host --rm aquasec/trivy --debug server --listen localhost:8080
2022-11-26T13:48:09.493Z        DEBUG   cache dir:  /home/runner/.cache/trivy
2022-11-26T13:48:09.493Z        DEBUG   There is no valid metadata file: unable to open a file: open /home/runner/.cache/trivy/db/metadata.json: no such file or directory
2022-11-26T13:48:09.493Z        INFO    Need to update DB
2022-11-26T13:48:09.493Z        INFO    DB Repository: ghcr.io/aquasecurity/trivy-db
2022-11-26T13:48:09.493Z        INFO    Downloading DB...
2022-11-26T13:48:09.493Z        DEBUG   no metadata file
2022-11-26T13:48:09.498Z        FATAL   failed to download vulnerability DB:
    github.com/aquasecurity/trivy/pkg/commands/operation.DownloadDB
        /home/runner/work/trivy/trivy/pkg/commands/operation/operation.go:117
  - OCI artifact error:
    github.com/aquasecurity/trivy/pkg/db.(*Client).Download
        /home/runner/work/trivy/trivy/pkg/db/db.go:154
  - OCI artifact error:
    github.com/aquasecurity/trivy/pkg/db.(*Client).initOCIArtifact
        /home/runner/work/trivy/trivy/pkg/db/db.go:194
  - OCI repository error:
    github.com/aquasecurity/trivy/pkg/oci.NewArtifact
        /home/runner/work/trivy/trivy/pkg/oci/artifact.go:69
  - Get "https://ghcr.io/v2/": dial tcp 140.82.121.34:443: connect: connection refused

Personally, I think it's related with the MITM stuff that proxy server is doing (see the sed I had run on debug1 output to make apk update work), but I'll test this hypothesis next week and come back with a follow-up.

danielnbalasoiu avatar Nov 26 '22 14:11 danielnbalasoiu

@danielnbalasoiu thanks for your help. i'll wait for your updates.

just a note. it doesn't matter in your case, but Trivy needs access to 2 hosts for DB downloading: ghcr.io and pkg-containers.githubusercontent.com

afdesk avatar Nov 26 '22 17:11 afdesk

@afdesk, I managed to get it working even without having to include --add-host parameter. It was a proxy problem related to the proxy config, so nothing has to be fixed on trivy side.

/ # trivy --debug server --listen localhost:8080
2022-11-29T16:49:33.345Z        DEBUG   cache dir:  /root/.cache/trivy
2022-11-29T16:49:33.345Z        DEBUG   There is no valid metadata file: unable to open a file: open /root/.cache/trivy/db/metadata.json: no such file or directory
2022-11-29T16:49:33.345Z        INFO    Need to update DB
2022-11-29T16:49:33.345Z        INFO    DB Repository: ghcr.io/aquasecurity/trivy-db
2022-11-29T16:49:33.345Z        INFO    Downloading DB...
2022-11-29T16:49:33.345Z        DEBUG   no metadata file
2022-11-29T16:49:36.180Z        DEBUG   Updating database metadata...
2022-11-29T16:49:36.180Z        DEBUG   DB Schema: 2, UpdatedAt: 2022-11-29 12:08:16.282509734 +0000 UTC, NextUpdate: 2022-11-29 18:08:16.282509334 +0000 UTC, DownloadedAt: 2022-11-29 16:49:36.180694584 +0000 UTC
2022-11-29T16:49:36.181Z        INFO    Listening localhost:8080...

Thank you for your prompt replies and your awesome work! 🚀

danielnbalasoiu avatar Nov 29 '22 16:11 danielnbalasoiu

This issue is stale because it has been labeled with inactivity.

github-actions[bot] avatar Feb 03 '23 00:02 github-actions[bot]

This issue is stale because it has been labeled with inactivity.

github-actions[bot] avatar Apr 10 '23 00:04 github-actions[bot]