libdnf icon indicating copy to clipboard operation
libdnf copied to clipboard

lockfile implementation causes pain for docker builds with cache mounts (unlike python-based dnf)

Open rmuir opened this issue 9 months ago • 2 comments

libdnf lockfile implementation has some "homemade" locking which doesn't play well with containerized environments where PIDs reproduce etc. It relies upon the existence of files and checking pids rather than using normal file-locking mechanisms.

In the docker world, users use "cache mounts" to share package cache across multiple containers. example:

RUN --mount=type=cache,target=/var/cache/yum,sharing=locked \
    microdnf install --setopt=keepcache=1 -y python3.11

It prevents downloading the same packages over and over again, especially with many containers. See https://docs.docker.com/build/cache/optimize/#use-cache-mounts for more information.

The bugs happen when user ^C's their build, which kills the microdnf process, but leaves a lockfile in the build cache. Unfortunately the next time they run docker build, they'll run microdnf, and it will have the same PID, and so it will inspect the file/pids and think they the lockfile is in use.

The issue only happens with microdnf, not dnf. python-based dnf uses flock()-based lock, so when the process dies, the lock is released, and such troubles don't happen.

rmuir avatar Jan 13 '25 19:01 rmuir