Reproducible Docker Builds

[chris-smith-zocdoc]

Is your feature request related to a problem? Please describe. Currently the docker_image target does not create a reproducible image, even when using pant's remote caching This creates problems for any dependent target that is referencing the docker digest, as those targets inputs will always change, preventing them from being catchable in our CI system.

The simplest version of this problem is related to timestamps of files as they are copied into the docker context. Since these files are copied into the pants sandbox first, the timestamps are always changing, which produces a new docker digest.

Describe the solution you'd like Buildkit 0.13.0 has made it very easy to re-write these timestamps in the image using the new rewrite-timestamp=true option

Setting the env var SOURCE_DATE_EPOCH also needs to occur so that the dates in the image metadata are stable. It would be nice if pants could derive this from the sources somehow, so that builds across different git commits are stable.

Additional context

Heres a script that showcases this behavior

#!/bin/bash
set -o errexit
set -o nounset
set -o pipefail
set -x trace


cat > Dockerfile.test <<EOF
FROM scratch

COPY test.txt /test.txt
EOF

if [ ! -f test.txt ]; then
    touch test.txt
fi

# buildx 0.13 is required for rewrite-timestamp
# https://github.com/moby/buildkit/blob/master/docs/build-repro.md
docker buildx create --use --driver-opt image=moby/buildkit:v0.13.1

build_opts="--tag=test:latest --output=type=docker,rewrite-timestamp=true --file=Dockerfile.test"

SOURCE_DATE_EPOCH=1710990413 docker buildx build $build_opts --iidfile=one.txt .

# remove all local caches/images to force a rebuild
docker system prune --all --force

sleep 1

SOURCE_DATE_EPOCH=1710990413 docker buildx build $build_opts --iidfile=two.txt .

if ! diff one.txt two.txt; then
  echo "Builds are not reproducible"
  exit 1
else
  echo "Builds are reproducible!"
fi

[kaos]

I imagine SOURCE_DATE_EPOCH could be equal to the commit date, in case the working dir is clean.