astral-sh
Docker: feedback on docs and workflow with frozen/locked dependencies and `--require-hashes`
Hello Astral team! Thank you for your work on Ruff and UV! It makes a huge difference.
Please clarify the best way to install the locked dependencies and validate by the hashes when building a Docker image.
Key points when building a Docker image of an app:
- We want minimum extra stuff inside the container.
- No need to create a venv.
- No need to install uv, because we can mount it (assume we use modern Docker with BuildKit).
- For production usage, we should use frozen/locked dependencies and validate them by hashes.
Problems:
uv synccreates a venv. This is redundant in a Docker container with a single app.- The
uv sync-way in Docker also requires usinguv runas cmd or setting the venv env var or override PATH. - The Docker section of the docs on the website seems to mix the new (UV-only) workflow and the old workflow when the UV is used as a faster Pip.
- It is unclear which way is preferred. A new user will be confused.
- Installing a project suggests doing
uv pip install -r pyproject.toml. But this way the lock/freeze files are not used and the build may be non-idempotent. And the hashes are not checked.
Workaround
This is what I ended up with right now with uv 0.3.1.
FROM python:3.12-slim as uv
# Installing via pip because there is no (yet) docker image for armv7
RUN --mount=type=cache,target=/root/.cache pip install -U uv
FROM python:3.12-slim
RUN mkdir /tmp/app
WORKDIR /tmp/app
COPY requirements.lock .
RUN --mount=type=cache,target=/root/.cache \
--mount=from=uv,source=/usr/local/bin/uv,target=/bin/uv \
# !!!! this grep hack !!!!
cat requirements.lock | grep -vE '^-e' > requirements.txt \
&& uv pip install --system --require-hashes --link-mode copy -r requirements.txt
What it should be like
I felt like I need something like uv sync —system --frozen-only which would not create a venv and would only use uv.lock to install the dependencies.
Versions
- uv 0.3.1
- Dev: MacOS Sonoma (14.3.1) on MacBook Pro with Apple Silicone arch.
- Prod: Ubuntu Linux 22.04 on armv7 arch.
There's something like this brewing in https://github.com/astral-sh/uv/pull/6398, designed for this very purpose (with discussion here: https://github.com/astral-sh/uv/issues/4028). It won't solve the --system part yet, that will come separately.
(Sorry, can't give a full reply now, I just wanted to point you there since it creates uv sync --frozen --no-locals to the project itself.)
You can run uv sync --frozen today though to install from only uv.lock.
You can run
uv sync --frozentoday though to install from onlyuv.lock.
Yes, but this way it creates a venv and installs in there. The desired way is to install into the system python in the docker container.
UPD1: and it also requires the pyproject.toml to be copied into the workdir.
Yeah the --no-locals from that PR would make it such that we don't require the pyproject.toml.
We don't yet support installing into system Python but I suspect we will soon for this reason.
There is a good point in https://github.com/astral-sh/uv/issues/4028#issuecomment-2305322593
uv sync is not a command designed for application install
What would be left after --no-locals is merged and the project deps are installed is that we'd still need to install the project itself (currently uv pip install .) which mixed the two worlds: pip and non-pip. We got ourselves another P vs NP problem.
Not quite -- you can see the example in the PR:
# Copy the lockfile into the image
ADD uv.lock /app/uv.lock
# Install remote dependencies
WORKDIR /app
RUN uv sync --frozen --no-locals
# Copy the project into the image
ADD . /app
WORKDIR /app
# Sync the remaining dependencies
RUN uv sync --frozen
The benefit here is that you can create a cacheable layer with your project's dependencies.
I could not find any issue that tracks the feature request of omitting venv creation so I will share another use case I found here (besides the Docker one).
Context
Mypy is notoriusly difficult to run correctly using pre-commit, since pre-commit uses an isolated env for every hook but mypy wants to have all your dependencies installed. To get around that, one can use language: system to omit venv creation and use the project's venv (with mypy and all dependencies installed):
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.6.1
hooks:
- id: mypy
language: system
Problem
In CI you need to install the dependencies to run mypy. If using uv, currently it looks something like this:
jobs:
pre-commit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version-file: ".python-version"
- name: Set up uv
run: curl -LsSf https://astral.sh/uv/0.3.1/install.sh | sh
- name: Install dependencies
run: uv sync --dev --frozen
- uses: pre-commit/[email protected]
However, that does not work because pre-commit uses the system Python and dependencies (and mypy) are installed into a venv.
Current solutions
As a workaround, you can either run pre-commit manually or add an extra step before pre-commit that adds the venv to the path and installs pip (needed for pre-commit to install the tools into the isolated venvs):
- run: |
echo "$PWD/.venv/bin" >> $GITHUB_PATH
uv pip install pip
The solution would be straightforward if a --system flag existed in uv sync.
EDIT: I've just found issue https://github.com/astral-sh/uv/issues/5964 that seems to be asking exactly for this
Furthermore, some Python libraries with native extensions must be built from sources, which require a compiler (or more than one, e.g. nvcc, rustc, etc). We want to keep that in the builder image only.
For now, we copy the .venv folder and have a workaround to fix the symlink:
RUN uv python install 3.10
RUN ln -sf $(uv python find 3.10) $HOME/$DIR_NAME/.venv/bin/python
It would be great to have a mechanism to "bundle" all the dependencies and python itself, so we could copy it to the runtime image and run the app without uv.
There's UV_PROJECT_ENVIRONMENT for omitting the virtual environment now. I'm not sure what else is actionable here.
