(WIP) Scan all tags for a Docker image

[rgmz]

Description:

This is a naive implementation that fixes #1753.

$ ./trufflehog docker --image=quay.io/fedora/fedora  --only-verified --all-tags 
🐷🔑🐷  TruffleHog. Unearth your secrets. 🐷🔑🐷

^[[6~2023-09-12T11:45:37-04:00  info-0  trufflehog      scanning image  {"source_id": 1, "job_id": 1, "source_manager_worker_id": "Jw91M", "source_type": "SOURCE_TYPE_DOCKER", "source_name": "trufflehog - docker", "source_type": "SOURCE_TYPE_DOCKER
", "source_name": "trufflehog - docker", "image": "quay.io/fedora/fedora", "tag": "36-aarch64"}
2023-09-12T11:45:43-04:00       info-0  trufflehog      scanning image  {"source_id": 1, "job_id": 1, "source_manager_worker_id": "Jw91M", "source_type": "SOURCE_TYPE_DOCKER", "source_name": "trufflehog - docker", "source_type": "SOURCE_TYPE_DOCKER
", "source_name": "trufflehog - docker", "image": "quay.io/fedora/fedora", "tag": "36-ppc64le"}
2023-09-12T11:45:49-04:00       info-0  trufflehog      scanning image  {"source_id": 1, "job_id": 1, "source_manager_worker_id": "Jw91M", "source_type": "SOURCE_TYPE_DOCKER", "source_name": "trufflehog - docker", "source_type": "SOURCE_TYPE_DOCKER
", "source_name": "trufflehog - docker", "image": "quay.io/fedora/fedora", "tag": "36-s390x"}

It is far from perfect, and requires feedback for considerations including:

• [ ] How to handle errors, specifically retryable (e.g. ratelimit) vs fatal, as the code currently fails fast
• [ ] How to handle older images, if at all (e.g. ubuntu:10.04) https://github.com/trufflesecurity/trufflehog/blob/8c6362925b9674cef8acdf1c4fec20b8cf155a62/pkg/sources/docker/docker.go#L199-L206
• [ ] How/if we should deduplicate scan results from layers that have already been scanned
• [ ] Sanity check for resource leaks?
• [ ] Test cases

Registry Implementations

This implementation has been tested on the following registries.

• [x] Artifactory (v7.59.0)
• [x] Docker Hub
• [x] Google Cloud Artifact Registry
• [ ] Nexus Repository 3
• [x] Quay.io

Checklist:

• [ ] Tests passing (make test-community)?
• [ ] Lint passing (make lint this requires golangci-lint)?

[dustin-decker]

Thanks for working on this contribution, @rgmz. One issue to consider here is that not all registry implementations support listing. I do not have a list right now, but I recall running into that previously. We need to make sure that the situation is handled gracefully.

[rgmz]

I do not have a list right now, but I recall running into that previously. We need to make sure that the situation is handled gracefully.

Good point.

I've added a list of registry implementations to the issue description. Let me know if you think of other implementations, or if you recall the specific one that caused issues.

[rgmz]

@dustin-decker I've refactored some of the logic. Let me know your thoughts on this so far.

Scanning one image

Old

$ ./trufflehog docker --image=ubuntu:10.04
🐷🔑🐷  TruffleHog. Unearth your secrets. 🐷🔑🐷

2023-10-21T14:17:14-04:00       error   trufflehog      scan errors     {"job_id": 1, "source_manager_worker_id": "TXPIz", "source_type": "SOURCE_TYPE_DOCKER", "source_name": "trufflehog - docker", "source_type": "SOURCE_TYPE_DOCKER", "source_name": "trufflehog - docker", "errors": "[unsupported MediaType: \"application/vnd.docker.distribution.manifest.v1+prettyjws\", see https://github.com/google/go-containerregistry/issues/377]"}
2023-10-21T14:17:14-04:00       info-0  trufflehog      finished scanning       {"chunks": 0, "bytes": 0, "verified_secrets": 0, "unverified_secrets": 0, "scan_duration": "654.20292ms"}

New

$ ./trufflehog docker --image=ubuntu:12.04
🐷🔑🐷  TruffleHog. Unearth your secrets. 🐷🔑🐷

2023-10-21T14:17:58-04:00       info-0  trufflehog      scanning image  {"job_id": 1, "source_manager_worker_id": "csVeY", "source_type": "SOURCE_TYPE_DOCKER", "source_name": "trufflehog - docker", "source_type": "SOURCE_TYPE_DOCKER", "source_name": "trufflehog - docker", "image": "index.docker.io/library/ubuntu:12.04"}
Found unverified result 🐷🔑❓
Detector Type: FixerIO
Decoder Type: PLAIN
Raw result: 2803709e26acf4303a1893252d34cceb
File: /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_precise_main_binary-amd64_Packages
Image: index.docker.io/library/ubuntu
Layer: sha256:d8868e50ac4c7104d2200d42f432b661b2da8c1e417ccfae217e6a1e04bb9295
Tag: 12.04

Scanning a local tarball

$ ./trufflehog docker --image=file:///tmp/fed.tar
🐷🔑🐷  TruffleHog. Unearth your secrets. 🐷🔑🐷

2023-10-21T14:29:37-04:00       info-0  trufflehog      scanning image  {"job_id": 1, "source_manager_worker_id": "yb5hY", "source_type": "SOURCE_TYPE_DOCKER", "source_name": "trufflehog - docker", "source_type": "SOURCE_TYPE_DOCKER", "source_name": "trufflehog - docker", "image": "file:///tmp/fed.tar"}
Found unverified result 🐷🔑❓
Detector Type: SQLServer
Decoder Type: BASE64
Raw result: = NULL || user != NULLauth_params == NULL || user != NULLInvalid IPv6 address  in URIIllegal encoded IP address  in URIIllegal internationalized hostname  in URICould not parse port  in URI in URI is out of range is not an absolute URI has no host componentbase_uri == NULL || base_uri->scheme != NULLURI is not absolute, and no base URI was providedMissing  and parameter ��nz�z�ڦ�bq�b��jg() called multiple timesg_set_user_dirs: Setting %s to %sCould not register atexit() function: %sgetpwuid_r(): failed due to unknown user id (%lu)getpwuid_r(): failed due to: %s.Could not find home directory:
File: /./usr/lib64/libglib-2.0.so.0.7800.0
Image: /tmp/fed.tar
Layer: sha256:0c963eefa3f26a27d3f4d535e58141a15ccda1d79894da07e8fd30706f72d236
...

Scanning --all-tags

$ ./trufflehog docker --image=ubuntu --all-tags
🐷🔑🐷  TruffleHog. Unearth your secrets. 🐷🔑🐷

2023-10-21T14:16:27-04:00       error   trufflehog      skipping unsupported v1 image    {"job_id": 1, "source_manager_worker_id": "auxkg", "source_type": "SOURCE_TYPE_DOCKER", "source_name": "trufflehog - docker", "source_type": "SOURCE_TYPE_DOCKER", "source_name": "trufflehog - docker", "image": "ubuntu", "error": "unsupported MediaType: \"application/vnd.docker.distribution.manifest.v1+prettyjws\", see https://github.com/google/go-containerregistry/issues/377"}
2023-10-21T14:16:28-04:00       info-0  trufflehog      scanning image  {"job_id": 1, "source_manager_worker_id": "auxkg", "source_type": "SOURCE_TYPE_DOCKER", "source_name": "trufflehog - docker", "source_type": "SOURCE_TYPE_DOCKER", "source_name": "trufflehog - docker", "image": "index.docker.io/library/ubuntu:12.04"}
Found unverified result 🐷🔑❓
Detector Type: FixerIO
Decoder Type: PLAIN
Raw result: 2803709e26acf4303a1893252d34cceb
File: /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_precise_main_binary-amd64_Packages
Image: index.docker.io/library/ubuntu
Layer: sha256:d8868e50ac4c7104d2200d42f432b661b2da8c1e417ccfae217e6a1e04bb9295
Tag: 12.04
...

Unsupported Options

$  ./trufflehog docker --image=ubuntu:12.04 --all-tags
🐷🔑🐷  TruffleHog. Unearth your secrets. 🐷🔑🐷

2023-10-21T16:22:47-04:00       error   trufflehog      engine failed to finish execution       {"error": "fatal: tag or digest can't be used with --all-tags (index.docker.io/library/ubuntu:12.04)"}

$ ./trufflehog docker --image=file:///tmp/fed.tar --all-tags
🐷🔑🐷  TruffleHog. Unearth your secrets. 🐷🔑🐷

2023-10-21T16:23:17-04:00       error   trufflehog      engine failed to finish execution       {"error": "fatal: --all-tags can't be used with a local file:// reference"}