Support authentication for --db-repository OCI registry

[blacksails]

We have a mirror of ghcr.io/aquasecurity/trivy-db our mirror requires authentication, but there doesnt seem to be any way to do authentication for the repository containing trivy-db.

➜ trivy image \
  --db-repository ourmirror.com/aquasecurity/trivy-db \
  someimage:latest
2022-06-21T15:52:16.580+0200    INFO    Need to update DB
2022-06-21T15:52:16.580+0200    INFO    DB Repository: ourmirror.com/aquasecurity/trivy-db
2022-06-21T15:52:16.580+0200    INFO    Downloading DB...
2022-06-21T15:52:16.743+0200    FATAL   init error: DB error: failed to download vulnerability DB: OCI artifact error: OCI artifact error: OCI repository error: GET https://ourmirror.com/v2/token?scope=repository%3Aaquasecurity%2Ftrivy-db%3Apull&service=ourmirror.com: : Authentication is required

The above has been adapted replacing our actual mirror with ourmirror.com. It would be nice if we could provide OCI authentication details.

[knqyf263]

Please try docker login.

[blacksails]

Please try docker login.

I am authenticated with docker

[knqyf263]

Hmm. It should work then. We use the same method as pulling private images. https://github.com/aquasecurity/trivy/blob/6b515bc73632e08a2d8d6d1bb9b8bea108c41fbe/pkg/oci/artifact.go#L59

Did you confirm you could pull another image from your private registry?

[blacksails]

Yes I tried replacing the above with

		remoteOpts := []remote.Option{
			remote.WithAuth(&authn.Basic{
				Username: os.Getenv("OCI_USER"),
				Password: os.Getenv("OCI_PASS"),
			}),
		}

Which seems to work. Maybe I should try to reproducce and raise it in the upstream google/go-containerregistry?

[knqyf263]

But it works in another place as I mentioned above. Could you write a small snippet to replicate it? If it works, it may be a Trivy issue.

[blacksails]

How does TRIVY_USERNAME and TRIVY_PASSWORD tie into this?

Do you only use the above authentication method for the database? eg. can I pull the database without the mirror and then scan an image on an authenticated registry without setting the TRIVY_USERNAME and TRIVY_PASSWORD variables?

[knqyf263]

TRIVY_USERNAME and TRIVY_PASSWORD are for pulling an image now, not the database.

[blacksails]

Is there any reason why it's not the same approach with regards to authentication?

[knqyf263]

Trivy cannot take two different credentials for database and image respectively via TRIVY_USERNAME and TRIVY_PASSWORD. If we want to support the approach, we may need TRIVY_DB_REPOSITORY_USERNAME or something like that. But I think docker login should satisfy most use cases for pulling database.

[thsbt]

Hi,

I face the same problem as @blacksails :

trivy image --db-repository  myrepo.com/aquasecurity/trivy-db --offline-scan myimage:tag

2022-06-22T13:22:29.641+0200	INFO	Need to update DB
2022-06-22T13:22:29.641+0200	INFO	DB Repository: myrepo.com/aquasecurity/trivy-db
2022-06-22T13:22:29.641+0200	INFO	Downloading DB...
2022-06-22T13:22:29.683+0200	FATAL	init error: DB error: failed to download vulnerability DB: OCI artifact error: OCI artifact error: OCI repository error: GET https://myrepo.com/v2/token?scope=repository%3Arepository%2Faquasecurity%2Ftrivy-db%3Apull&service=myrepo.com: : Authentication is required

I'm well authenticated against myrepo.com as I can pull docker images from it. Moreover, I can pull the trivy-db via oras without any problem :

oras pull myrepo.com/aquasecurity/trivy-db:2 -a

Downloaded 5affceed0f4b db.tar.gz
Pulled myrepo.com/aquasecurity/trivy-db:2
Digest: sha256:a41...

I'm running trivy v0.28.1 and docker 20.10.9 but in rootless mode. May the problem comes from the rootless mode ?

[sys-ops]

Hi Y'All, Unfortunately, I started facing the same issue today. Yesterday everything was fine. I use private docker registry but the default OCI repository to retrieve trivy-db from "ghcr.io/aquasecurity/trivy-db".

2022-06-22T14:26:33.920+0200    INFO    Need to update DB
2022-06-22T14:26:33.920+0200    INFO    DB Repository: ghcr.io/aquasecurity/trivy-db
2022-06-22T14:26:33.920+0200    INFO    Downloading DB...
2022-06-22T14:26:35.099+0200    FATAL   DB error: failed to download vulnerability DB: database download error: failed to fetch the layer: GET https://ghcr.io/v2/aquasecurity/trivy-db/blobs/sha256:650e60eb07987f7716e708acd0a6ec305018e134175bf09e6d9162e75cb9dc91: UNAUTHORIZED: unauthenticated: User cannot be authenticated with the token provided.

I did not upgrade trivy. I use version 0.25.4. Why should I use TRIVY_USERNAME and TRIVY_PASSWORD?

[knqyf263]

@sys-ops Your issue is due to GitHub being down. Please refer https://github.com/aquasecurity/trivy/issues/2385.

[knqyf263]

@ThomasBertiere Thanks for updating. We don't use Docker Engine, but a credential. Rootless should not matter.

@afdesk Could you replicate it? Upload trivy-db to your private repository in GHCR, set up a credential and try --db-repository.

[afdesk]

I've tried to reproduce it. my steps:

$ oras pull ghcr.io/aquasecurity/trivy-db:2 -a

$ export CR_PAT=ghp_MY_GITHUB_TOKEN
$ echo $CR_PAT | oras login https://ghcr.io -u <MY GITHUB ACC> --password-stdin

$ oras push ghcr.io/<MY GITHUB ACC>/trivy-db-private:2 db.tar.gz

and I get an error when I run trivy:

$ trivy i --db-repository ghcr.io/<MY GITHUB ACC>/trivy-db-private alpine:latest
...
2022-06-22T21:47:48.044+0600	FATAL	init error: DB error: failed to download vulnerability DB: OCI artifact error: OCI artifact error: unacceptable media type: application/vnd.oci.image.layer.v1.tar

but it's another issue... will take a look more

[knqyf263]

@afdesk You have to pass a media type. https://github.com/aquasecurity/trivy-db/blob/39d8a6798e072884afa497c4939036dbafd3b2e4/.github/workflows/cron.yml#L56-L58

[afdesk]

yes... it works:

$ oras push ghcr.io/<MY GITHUB ACC>/trivy-db-private:2 db.tar.gz:application/vnd.aquasec.trivy.db.layer.v1.tar+gzip

[afdesk]

$ trivy i --db-repository ghcr.io/<MY GITHUB ACC>/trivy-db-private alpine:latest
2022-06-22T22:12:22.348+0600	INFO	Need to update DB
2022-06-22T22:12:22.348+0600	INFO	DB Repository: ghcr.io/<MY GITHUB ACC>/trivy-db-private
2022-06-22T22:12:22.348+0600	INFO	Downloading DB...
32.65 MiB / 32.65 MiB [-----------------------------------------------------------------------------------------------------] 100.00% 3.45 MiB p/s 9.7s
2022-06-22T22:12:33.743+0600	INFO	Vulnerability scanning is enabled

[afdesk]

I've removed my access token from GHCR and could get a similar error:

$ trivy i --db-repository ghcr.io/<MY GITHUB ACC>/trivy-db-private alpine:latest
2022-06-22T22:25:02.556+0600	INFO	Need to update DB
2022-06-22T22:25:02.556+0600	INFO	DB Repository: ghcr.io/<MY GITHUB ACC>/trivy-db-private
2022-06-22T22:25:02.557+0600	INFO	Downloading DB...
2022-06-22T22:25:03.180+0600	FATAL	init error: DB error: failed to download vulnerability DB: OCI artifact error: OCI artifact error: OCI repository error: GET https://ghcr.io/token?scope=repository%3A<MY GITHUB ACC>%2Ftrivy-db-private%3Apull&service=ghcr.io: DENIED: denied

[afdesk]

I created a new token and login via docker. it also works for me:

$ export CR_PAT=ghp_NEW_TOKEN
$ echo $CR_PAT | docker login ghcr.io -u USERNAME --password-stdin
WARNING! Your password will be stored unencrypted in ________
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

$ trivy i --db-repository ghcr.io/<MY GITHUB ACC>/trivy-db-private alpine:latest
2022-06-22T22:34:56.905+0600	INFO	Need to update DB
2022-06-22T22:34:56.905+0600	INFO	DB Repository: ghcr.io/<MY GITHUB ACC>/trivy-db-private
2022-06-22T22:34:56.905+0600	INFO	Downloading DB...
32.65 MiB / 32.65 MiB [------------------------------------------------------------------------------------------------------] 100.00% 2.59 MiB p/s 13s
2022-06-22T22:35:11.370+0600	INFO	Vulnerability scanning is enabled

[afdesk]

I have no idea what else to try

[knqyf263]

@afdesk Thanks for testing. Yes, it should work as we expected. It is weird that oras works, but Trivy doesn't. macOS vs Linux issue?

[afdesk]

oh, i can try to test on MacOs

[afdesk]

tested. it works correctly on macOS Monterey 12.1

[afdesk]

maybe the custom servers have specific permissions...

[knqyf263]

Hmm. I have no idea why it doesn't work then...

[thsbt]

Hi,

I tried a second repository (following what @afdesk did) and it works well !

$ oras pull myFirstRepo.com/aquasecurity/trivy-db:2 -a
$ oras login mySecondRepo.com
$ oras push mySecondRepo.com/aquasecurity/trivy-db:2 -a db.tar.gz:application/vnd.aquasec.trivy.db.layer.v1.tar+gzip

$ trivy image --db-repository mySecondRepo.com/aquasecurity/trivy-db --offline-scan myimage:tag

2022-06-23T09:26:40.757+0200	[34mINFO[0m	Need to update DB
2022-06-23T09:26:40.757+0200	[34mINFO[0m	DB Repository: mySecondRepo.com/aquasecurity/trivy-db
2022-06-23T09:26:40.757+0200	[34mINFO[0m	Downloading DB...
2022-06-23T09:26:40.757+0200	[34mINFO[0m	32.67 MiB / 32.67 MiB [------------------------------------------------------------------------------------------------------------] 100.00% 1.09 MiB p/s 30s
...

But it still not works with my first repository. My first repository is a JFrog Artifactory miroring ghcr.io. My second respository is an Harbor repository.

[knqyf263]

@ThomasBertiere Thanks for confirming. I don't have an environment to test with JFrog Artifactory, but does oras work for example? It should work if JFrog Artifactory is a OCI registry.

[afdesk]

@ThomasBertiere is there anything strang in the JFrog logs?

[thsbt]

Ok I made a mistake... 😩

My second repository (Harbor) was Public so no need of authentication. I change it to Private and trivy does not work. The error message is slightly different :

$ trivy image --db-repository mySecondRepo.com/aquasecurity/trivy-db --offline-scan myimage:tag
 
2022-06-23T14:28:04.908+0200	INFO	Need to update DB
2022-06-23T14:28:04.908+0200	INFO	DB Repository: mySecondRepo.com/aquasecurity/trivy
2022-06-23T14:28:04.908+0200	INFO	Downloading DB...
2022-06-23T14:28:05.001+0200	FATAL	init error: DB error: failed to download vulnerability DB: OCI artifact error: OCI artifact error: OCI repository error: GET https://mySecondRepo.com/v2/aquasecurity/trivy/manifests/2: UNAUTHORIZED: unauthorized to access repository: aquasecurity/trivy, action: pull: unauthorized to access repository: aquasecurity/trivy, action: pull

But I can pull it via oras :

oras pull mySecondRepo.com/aquasecurity/trivy:2 -a

Downloaded fe21c8f4550b db.tar.gz
Pulled mySecondRepo.com/aquasecurity/trivy:2
Digest: sha256:bcd0f...

And I can pull docker images from mySecondRepo.com.

@afdesk unfortunately I do not have access to JFrog logs...

[afdesk]

@ThomasBertiere oh, thanks. will try again.