trufflesecurity
Scan docker image config metadata
Description:
This is my attempt at solving #3007
The current implementation only scans the created_by value in the docker image config (https://github.com/opencontainers/image-spec/blob/main/config.md). This misses e.g. the environment variables of the docker image and other fields that commonly contain secrets.
The easiest way to view this metadata is by using "crane config [image]" https://github.com/google/go-containerregistry/tree/main/cmd/crane
Checklist:
- [x] Tests passing (
make test-community)? (fails but also fails on main without my changes) - [x] Lint passing (
make lintthis requires golangci-lint)?
Hey, sorry for letting this fall through the cracks. I'm no docker expert - does this PR preserve the existing behavior of scanning each history entry's createdBy?
Hey, sorry for letting this fall through the cracks. I'm no docker expert - does this PR preserve the existing behavior of scanning each history entry's
createdBy?
Yes, it scans the whole config file that contains the createdBy entry's
Here is an example of what the nginx conf looks like from crane config nginx | jq, note the "created_by" fields
Nginx docker conf
{
"architecture": "amd64",
"config": {
"ExposedPorts": {
"80/tcp": {}
},
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"NGINX_VERSION=1.27.4",
"NJS_VERSION=0.8.9",
"NJS_RELEASE=1~bookworm",
"PKG_RELEASE=1~bookworm",
"DYNPKG_RELEASE=1~bookworm"
],
"Entrypoint": [
"/docker-entrypoint.sh"
],
"Cmd": [
"nginx",
"-g",
"daemon off;"
],
"Labels": {
"maintainer": "NGINX Docker Maintainers <[email protected]>"
},
"StopSignal": "SIGQUIT"
},
"created": "2025-02-05T21:27:16Z",
"history": [
{
"created": "2025-02-05T21:27:16Z",
"created_by": "# debian.sh --arch 'amd64' out/ 'bookworm' '@1740355200'",
"comment": "debuerreotype 0.15"
},
{
"created": "2025-02-05T21:27:16Z",
"created_by": "LABEL maintainer=NGINX Docker Maintainers <[email protected]>",
"comment": "buildkit.dockerfile.v0",
"empty_layer": true
},
{
"created": "2025-02-05T21:27:16Z",
"created_by": "ENV NGINX_VERSION=1.27.4",
"comment": "buildkit.dockerfile.v0",
"empty_layer": true
},
{
"created": "2025-02-05T21:27:16Z",
"created_by": "ENV NJS_VERSION=0.8.9",
"comment": "buildkit.dockerfile.v0",
"empty_layer": true
},
{
"created": "2025-02-05T21:27:16Z",
"created_by": "ENV NJS_RELEASE=1~bookworm",
"comment": "buildkit.dockerfile.v0",
"empty_layer": true
},
{
"created": "2025-02-05T21:27:16Z",
"created_by": "ENV PKG_RELEASE=1~bookworm",
"comment": "buildkit.dockerfile.v0",
"empty_layer": true
},
{
"created": "2025-02-05T21:27:16Z",
"created_by": "ENV DYNPKG_RELEASE=1~bookworm",
"comment": "buildkit.dockerfile.v0",
"empty_layer": true
},
{
"created": "2025-02-05T21:27:16Z",
"created_by": "RUN /bin/sh -c set -x && groupadd --system --gid 101 nginx && useradd --system --gid nginx --no-create-home --home /nonexistent --comment \"nginx user\" --shell /bin/false --uid 101 nginx && apt-get update && apt-get install --no-install-recommends --no-install-suggests -y gnupg1 ca-certificates && NGINX_GPGKEYS=\"573BFD6B3D8FBC641079A6ABABF5BD827BD9BF62 8540A6F18833A80E9C1653A42FD21310B49F6B46 9E9BE90EACBCDE69FE9B204CBCDCD8A38D88A2B3\"; NGINX_GPGKEY_PATH=/etc/apt/keyrings/nginx-archive-keyring.gpg; export GNUPGHOME=\"$(mktemp -d)\"; found=''; for NGINX_GPGKEY in $NGINX_GPGKEYS; do for server in hkp://keyserver.ubuntu.com:80 pgp.mit.edu ; do echo \"Fetching GPG key $NGINX_GPGKEY from $server\"; gpg1 --keyserver \"$server\" --keyserver-options timeout=10 --recv-keys \"$NGINX_GPGKEY\" && found=yes && break; done; test -z \"$found\" && echo >&2 \"error: failed to fetch GPG key $NGINX_GPGKEY\" && exit 1; done; gpg1 --export \"$NGINX_GPGKEYS\" > \"$NGINX_GPGKEY_PATH\" ; rm -rf \"$GNUPGHOME\"; apt-get remove --purge --auto-remove -y gnupg1 && rm -rf /var/lib/apt/lists/* && dpkgArch=\"$(dpkg --print-architecture)\" && nginxPackages=\" nginx=${NGINX_VERSION}-${PKG_RELEASE} nginx-module-xslt=${NGINX_VERSION}-${DYNPKG_RELEASE} nginx-module-geoip=${NGINX_VERSION}-${DYNPKG_RELEASE} nginx-module-image-filter=${NGINX_VERSION}-${DYNPKG_RELEASE} nginx-module-njs=${NGINX_VERSION}+${NJS_VERSION}-${NJS_RELEASE} \" && case \"$dpkgArch\" in amd64|arm64) echo \"deb [signed-by=$NGINX_GPGKEY_PATH] https://nginx.org/packages/mainline/debian/ bookworm nginx\" >> /etc/apt/sources.list.d/nginx.list && apt-get update ;; *) tempDir=\"$(mktemp -d)\" && chmod 777 \"$tempDir\" && savedAptMark=\"$(apt-mark showmanual)\" && apt-get update && apt-get install --no-install-recommends --no-install-suggests -y curl devscripts equivs git libxml2-utils lsb-release xsltproc && ( cd \"$tempDir\" && REVISION=\"${NGINX_VERSION}-${PKG_RELEASE}\" && REVISION=${REVISION%~*} && curl -f -L -O https://github.com/nginx/pkg-oss/archive/${REVISION}.tar.gz && PKGOSSCHECKSUM=\"973690e64fa47e3704e817a3b08205b9e3f8c0cbe31825d9d62a81c11eb3aa186df015f27fdfd48c8799ffc528e38a9168c592ae665e4835c2d28638ec5f7845 *${REVISION}.tar.gz\" && if [ \"$(openssl sha512 -r ${REVISION}.tar.gz)\" = \"$PKGOSSCHECKSUM\" ]; then echo \"pkg-oss tarball checksum verification succeeded!\"; else echo \"pkg-oss tarball checksum verification failed!\"; exit 1; fi && tar xzvf ${REVISION}.tar.gz && cd pkg-oss-${REVISION} && cd debian && for target in base module-geoip module-image-filter module-njs module-xslt; do make rules-$target; mk-build-deps --install --tool=\"apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends --yes\" debuild-$target/nginx-$NGINX_VERSION/debian/control; done && make base module-geoip module-image-filter module-njs module-xslt ) && apt-mark showmanual | xargs apt-mark auto > /dev/null && { [ -z \"$savedAptMark\" ] || apt-mark manual $savedAptMark; } && ls -lAFh \"$tempDir\" && ( cd \"$tempDir\" && dpkg-scanpackages . > Packages ) && grep '^Package: ' \"$tempDir/Packages\" && echo \"deb [ trusted=yes ] file://$tempDir ./\" > /etc/apt/sources.list.d/temp.list && apt-get -o Acquire::GzipIndexes=false update ;; esac && apt-get install --no-install-recommends --no-install-suggests -y $nginxPackages gettext-base curl && apt-get remove --purge --auto-remove -y && rm -rf /var/lib/apt/lists/* /etc/apt/sources.list.d/nginx.list && if [ -n \"$tempDir\" ]; then apt-get purge -y --auto-remove && rm -rf \"$tempDir\" /etc/apt/sources.list.d/temp.list; fi && ln -sf /dev/stdout /var/log/nginx/access.log && ln -sf /dev/stderr /var/log/nginx/error.log && mkdir /docker-entrypoint.d # buildkit",
"comment": "buildkit.dockerfile.v0"
},
{
"created": "2025-02-05T21:27:16Z",
"created_by": "COPY docker-entrypoint.sh / # buildkit",
"comment": "buildkit.dockerfile.v0"
},
{
"created": "2025-02-05T21:27:16Z",
"created_by": "COPY 10-listen-on-ipv6-by-default.sh /docker-entrypoint.d # buildkit",
"comment": "buildkit.dockerfile.v0"
},
{
"created": "2025-02-05T21:27:16Z",
"created_by": "COPY 15-local-resolvers.envsh /docker-entrypoint.d # buildkit",
"comment": "buildkit.dockerfile.v0"
},
{
"created": "2025-02-05T21:27:16Z",
"created_by": "COPY 20-envsubst-on-templates.sh /docker-entrypoint.d # buildkit",
"comment": "buildkit.dockerfile.v0"
},
{
"created": "2025-02-05T21:27:16Z",
"created_by": "COPY 30-tune-worker-processes.sh /docker-entrypoint.d # buildkit",
"comment": "buildkit.dockerfile.v0"
},
{
"created": "2025-02-05T21:27:16Z",
"created_by": "ENTRYPOINT [\"/docker-entrypoint.sh\"]",
"comment": "buildkit.dockerfile.v0",
"empty_layer": true
},
{
"created": "2025-02-05T21:27:16Z",
"created_by": "EXPOSE map[80/tcp:{}]",
"comment": "buildkit.dockerfile.v0",
"empty_layer": true
},
{
"created": "2025-02-05T21:27:16Z",
"created_by": "STOPSIGNAL SIGQUIT",
"comment": "buildkit.dockerfile.v0",
"empty_layer": true
},
{
"created": "2025-02-05T21:27:16Z",
"created_by": "CMD [\"nginx\" \"-g\" \"daemon off;\"]",
"comment": "buildkit.dockerfile.v0",
"empty_layer": true
}
],
"os": "linux",
"rootfs": {
"type": "layers",
"diff_ids": [
"sha256:5f1ee22ffb5e68686db3dcb6584eb1c73b5570615b0f14fabb070b96117e351d",
"sha256:c68632c455ae0c46d1380033bae6d30014853fa3f600f4e14efc440be1bc9580",
"sha256:cabea05c000e49f0814b2611cbc66c2787f609d8a27fc7b9e97b5dab5d8502da",
"sha256:791f0a07985c2814a899cb0458802be06ba124a364f7e5a9413a1f08fdbf5b5c",
"sha256:f6d5815f290ee912fd4a768d97b46af39523dff584d786f5c0f7e9bdb7fad537",
"sha256:7d22e2347c1217a89bd3c79ca9adb4652c1e9b61427fffc0ab92227aacd19a38",
"sha256:55e9644f21c38d7707b4a432aacc7817c5414b68ac7a750e704c2f7100ebc15c"
]
}
}
Ok, thanks! I do see that you've removed the Layer metadata entry from these secrets. That makes total sense, but unfortunately, removing metadata is something we try really hard to avoid because anyone using TruffleHog as part of a workflow that persistently tracks secrets will see metadata change, which can cause very irritating problems. This unfortunately isn't esoteric - it's every user of TruffleHog Enterprise 😞
How much work would it be to somehow retrieve layer information for each of these found secrets so that that metadata field doesn't change?
Also, we should use a ChunkReader to scan the config file in case it gets really big. (The docker source already does this for the layers themselves.)
Ok, thanks! I do see that you've removed the
Layermetadata entry from these secrets. That makes total sense, but unfortunately, removing metadata is something we try really hard to avoid because anyone using TruffleHog as part of a workflow that persistently tracks secrets will see metadata change, which can cause very irritating problems. This unfortunately isn't esoteric - it's every user of TruffleHog Enterprise 😞How much work would it be to somehow retrieve layer information for each of these found secrets so that that metadata field doesn't change?
I understand, I'll see if I can come up with something, though I will probably have to parse the json file then. However will be a few weeks before I can look into it