uv icon indicating copy to clipboard operation
uv copied to clipboard

Feature request: Generate `requirements.txt` from `uv.lock`

Open zeevro opened this issue 1 year ago • 18 comments

It would be beneficial to me (and I think in general) to be able to generate a requirements.txt file from the uv.lock file. It would be useful for installing requirements without having to resolve dependencies using a different installer in environments where uv is not installed, and also for having other tools that don't support the uv.lock file format (e.g. Dependabot) be able to know the project's pinned dependencies.

zeevro avatar Aug 11 '24 14:08 zeevro

I think something like this is reasonable but it probably won't happen immediately. (PEP 751 is ultimately a better solution but I'd still support requirements.txt in the short-term.)

charliermarsh avatar Aug 11 '24 18:08 charliermarsh

Also, we already have a way to generate requirements.txt in the form of uv pip compile but it's rather tricky to have that match my uv.lock, especially if it's done after a while and some dependencies have had new versions released.

zeevro avatar Aug 12 '24 07:08 zeevro

It doesn't really seem like a great outcome for users to have both a uv.lock and requirements.txt though

zanieb avatar Aug 12 '24 12:08 zanieb

It doesn't really seem like a great outcome for users to have both a uv.lock and requirements.txt though

If using uv sync which creates a uv.lock file in a Docker, does uv sync take care of setting up references to the venv correctly ? https://docs.astral.sh/uv/guides/integration/docker/#installing-a-package makes no mention of uv sync or uv.lock

SamuelT-Beslogic avatar Aug 14 '24 18:08 SamuelT-Beslogic

Can you clarify what you mean by "references to the venv"?

I haven't written documentation for using the new APIs in Docker yet.

zanieb avatar Aug 14 '24 19:08 zanieb

Can you clarify what you mean by "references to the venv"?

I'm talking about this section of the doc:

To use the system Python environment by default, set the UV_SYSTEM_PYTHON variable:

ENV UV_SYSTEM_PYTHON=1

Alternatively, a virtual environment can be created and activated:

RUN uv venv /opt/venv
# Use the virtual environment automatically
ENV VIRTUAL_ENV=/opt/venv
# Place entry points in the environment at the front of the path
ENV PATH="/opt/venv/bin:$PATH"

I'm assuming if I use RUN uv sync --locked instead (to make sure the installation respects the lock file), that I don't need to mess around with venv setup ? (I just need to update my docker entry point command to use uv run instead of python, ref to our current transition to UV https://github.com/BesLogic/releaf-canopeum/pull/195/files#diff-1fc9c5328b5af6b1065537cb89aa7298eb7ba1d338b5bee8fd07553b6f56576f )

I haven't written documentation for using the new APIs in Docker yet.

That's a simple explanation as to why it's not referenced ^^


I'm just ensuring that the uv.lock file should be usable in Docker. Which removes the need for this feature request for me.

SamuelT-Beslogic avatar Aug 14 '24 19:08 SamuelT-Beslogic

uv sync always creates a virtual environment right now, so yeah you'd either need to opt for configuring it (as described above) or use uv run instead of invoking python.

We may add support for syncing to a system environment (as requested in https://github.com/astral-sh/uv/issues/5964) to make this a little simpler in CI and containers.

I'm just ensuring that the uv.lock file should be usable in Docker

Yeah it definitely is. I'll try to update the documentation soon.

zanieb avatar Aug 14 '24 19:08 zanieb

It doesn't really seem like a great outcome for users to have both a uv.lock and requirements.txt though

I have this use case right now; maybe the insight is useful. There are strict requirements only to deploy stable software in the deployment targets. That rules out uv on the deployment targets by our current rules. The devs still want to use uv and uv project management and it is pretty confusing to have the requirements.txt that is used for deployments and the uv.lock within the repo.

This can be solved by CI/CD but reducing steps and complexity is the signature of uv/ruff for me. Loving the tools btw, y'all are doing gods work

matmair avatar Aug 20 '24 19:08 matmair

What makes uv unstable?

Thanks for sharing your use-case! Glad you like the tools :)

zanieb avatar Aug 20 '24 19:08 zanieb

SemVer. I do not make the rules sadly.

matmair avatar Aug 20 '24 19:08 matmair

I also am finding a need to produce a requirements.txt, but just for non-dev packages. The current path I have in our pipeline looks something like:

uv sync --no-dev
uv pip freeze > requirements.txt

Then we can check that requirements file against safety (or dependabot) for vulnerabilities, then do a uv pip install -r requirements.txt --target dist so we can package it up along with src into a zip file to deploy to a lambda function.

It definitely would be easier to operate directly against the uv.lock, but in the meantime converting that directly into a requirements.txt for compatibility with other tools would be great.

jfgordon2 avatar Aug 22 '24 05:08 jfgordon2

+1 to what @jfgordon2 said.

I think supporting requirements.txt input and output is wise to maintain compatibility with legacy tooling that isn’t yet caught up with PEP 621.

I have documented one such use case for requirements.txt here: #6422

chrisrodrigue avatar Aug 22 '24 11:08 chrisrodrigue

As you mentioned in #6429 , will there be a feature to create a requirements.txt including tool.uv.dev-dependencies?

phi-friday avatar Aug 26 '24 02:08 phi-friday

Yeah, that would probably be included in the scope of this.

charliermarsh avatar Aug 26 '24 02:08 charliermarsh

I'm using this as a make-shift solution for now.

uvx --from=yq tomlq -r '.package[]|.name+((.source|" @ "+(.url//"git+\(.git//empty)"))//"==\(.version)")' uv.lock >requirements.txt

zeevro avatar Aug 26 '24 19:08 zeevro

Respect!

charliermarsh avatar Aug 26 '24 20:08 charliermarsh

I like the idea of uv export!

I am curious what are the implications of doing this:

uv pip compile pyproject.toml -o requirements.txt

In this case is my uv.lock ignored and there is a risk the generated requirements.txt will be different?

SamEdwardes avatar Aug 28 '24 22:08 SamEdwardes

In this case is my uv.lock ignored and there is a risk the generated requirements.txt will be different?

Yeah, the uv pip interface never touches the lockfile. You can use the uv pip workflow if you want, but you lose the benefits of the unified uv sync, uv lock, uv run APIs.

charliermarsh avatar Aug 28 '24 22:08 charliermarsh

Curious if anyone has tips / tools they like to keep requirements.txt in sync with uv.lock so you get the benefits of uv while still being easy to use for code reviewers. Even though I know it is recommended to just use uv, I develop libraries where it is easier to team members to quickly use pip and might not be familiar with uv.

I am just using a simple github action but wasnt sure if there is a better way.

name: Validate Requirements

on:
  workflow_dispatch:
  push:
    paths:
      - "pyproject.toml"  # Trigger on changes to pyproject.toml
      - "requirements.txt"
  pull_request:
    paths:
      - "pyproject.toml"  # Trigger on changes to pyproject.toml
      - "requirements.txt"

jobs:
  validate-requirements:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4


      - name: Install uv
        uses: astral-sh/setup-uv@v4

      - name: "Set up Python"
        uses: actions/setup-python@v5
        with:
          python-version-file: ".python-version"

      - name: Install the project
        run: uv sync --all-extras --dev
        
      - name: Generate new requirements.txt
        run: |
          uv export --format requirements-txt > new_requirements.txt

      - name: Compare requirements.txt
        run: |
          if ! diff -q new_requirements.txt requirements.txt; then
            echo "requirements.txt is out of sync with the project configuration:"
            diff -u requirements.txt new_requirements.txt
            exit 1
          fi
        shell: bash

C-Loftus avatar Dec 16 '24 15:12 C-Loftus

There's also support in the uv-pre-commit project.

zanieb avatar Dec 16 '24 19:12 zanieb

@C-Loftus,

name: Validate Requirements

on:
  workflow_dispatch:
  push:
    paths:
      - "pyproject.toml"  # Trigger on changes to pyproject.toml
      - "requirements.txt"
  pull_request:
    paths:
      - "pyproject.toml"  # Trigger on changes to pyproject.toml
      - "requirements.txt"

Shouldn't you also validate on changes to uv.lock? It's possible to do a dependency package update without touching pyproject.toml, after all.

leorochael avatar Jan 21 '25 11:01 leorochael

Curious if anyone has tips / tools they like to keep requirements.txt in sync with uv.lock so you get the benefits of uv while still being easy to use for code reviewers. Even though I know it is recommended to just use uv, I develop libraries where it is easier to team members to quickly use pip and might not be familiar with uv.

I am just using a simple github action but wasnt sure if there is a better way.

name: Validate Requirements

on: workflow_dispatch: push: paths: - "pyproject.toml" # Trigger on changes to pyproject.toml - "requirements.txt" pull_request: paths: - "pyproject.toml" # Trigger on changes to pyproject.toml - "requirements.txt"

jobs: validate-requirements: runs-on: ubuntu-latest

steps:
  - name: Checkout repository
    uses: actions/checkout@v4


  - name: Install uv
    uses: astral-sh/setup-uv@v4

  - name: "Set up Python"
    uses: actions/setup-python@v5
    with:
      python-version-file: ".python-version"

  - name: Install the project
    run: uv sync --all-extras --dev
    
  - name: Generate new requirements.txt
    run: |
      uv export --format requirements-txt > new_requirements.txt

  - name: Compare requirements.txt
    run: |
      if ! diff -q new_requirements.txt requirements.txt; then
        echo "requirements.txt is out of sync with the project configuration:"
        diff -u requirements.txt new_requirements.txt
        exit 1
      fi
    shell: bash

Are you sure that this is not outdated now? @C-Loftus

simonprovost avatar Apr 11 '25 01:04 simonprovost

uv export --no-emit-workspace --no-dev --no-annotate --no-header --no-hashes --output-file requirements.txt thanks for the implement, this worked for me!

codejoey avatar May 26 '25 00:05 codejoey

(talitrain) root@aa0d5ec39dd8:~/talitrain# uv export --no-emit-workspace --no-dev --no-annotate --no-header --no-hashes --output-file requirements.txt
error: unexpected argument '--no-annotate' found

Usage: uv export --no-emit-workspace --no-dev

For more information, try '--help'.

I am sticking with: uv pip freeze > requirements.txt

ilovefreesw avatar Jul 03 '25 07:07 ilovefreesw

(talitrain) root@aa0d5ec39dd8:~/talitrain# uv export --no-emit-workspace --no-dev --no-annotate --no-header --no-hashes --output-file requirements.txt
error: unexpected argument '--no-annotate' found

Usage: uv export --no-emit-workspace --no-dev

For more information, try '--help'.

I am sticking with: uv pip freeze > requirements.txt

your uv version might be outdated.

vesnikos avatar Sep 19 '25 09:09 vesnikos