ocaml-dockerfile icon indicating copy to clipboard operation
ocaml-dockerfile copied to clipboard

Cache distribution package downloads with BuildKit cache mounts

Open MisterDA opened this issue 1 year ago • 2 comments

I think sharing the package cache could make docker builds more efficient, but I'm also worried that parallel jobs could compete for the cache as it is exclusive (locked). Alternatively, it could be made private (creates a new mount if there are multiple writers), see RUN --mount=type=cache.

The code comes from an example in the Docker docs Example: cache apt packages:

# syntax=docker/dockerfile:1
FROM ubuntu
RUN rm -f /etc/apt/apt.conf.d/docker-clean; echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/keep-cache
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
  --mount=type=cache,target=/var/lib/apt,sharing=locked \
  apt update && apt-get --no-install-recommends install -y gcc

Apt needs exclusive access to its data, so the caches use the option sharing=locked, which will make sure multiple parallel builds using the same cache mount will wait for each other and not access the same cache files at the same time. You could also use sharing=private if you prefer to have each build create another cache directory in this case.

MisterDA avatar Nov 06 '24 09:11 MisterDA