pip-tools icon indicating copy to clipboard operation
pip-tools copied to clipboard

Generate both the requirements file and the corresponding contraints file in one command

Open nabla-c0d3 opened this issue 2 years ago • 1 comments

What's the problem this feature will solve?

Our application uses two sets of requirements:

  • requirements.in
  • requirements-dev.in

Previously, we would add -c requirements.txt at the top of our requirements-dev.in to ensure that compatible packages get installed, as described in the "Workflow for layered requirements" section of pip-tools' README.

Since pip changed how -c works by restricting what's allowed in "constraints files" (https://pip.pypa.io/en/stable/user_guide/#constraints-files), we can no longer put -c requirements.txt at the top of our requirements-dev.in.

This is because our requirements.txt contains entries that are not allowed to be in a constraints file, including extras and file: (ie. vendored) dependencies.

Describe the solution you'd like

One solution would be to have a command line option in pip-tools to generate a valid constraints file in addition to the requirements file. Something like this:

pip-compile --resolver=backtracking requirements.in -o requirements.txt --output-as-constraint-file requirements-as-constraints.txt

We would then be able to add -c requirements-as-constraints.txt to the top of our requirements-dev.in and have a similar workflow as before. Having both the requirements.txt and requirements-as-constraints.txt files generated in one command would also ensure that they are in sync.

Alternative Solutions

I couldn't find a simple workflow for what we're trying to do . pip-tools has --strip-extras but this doesn't help for file: dependencies, and it also would require an extra command to generate a constraints file.

Our current approach, which is unideal, is to just copy and paste problematic entries from the requirements.in to the requirement-dev.in.

nabla-c0d3 avatar Jan 31 '23 13:01 nabla-c0d3

I discovered today another reason to support this: invoking pip-compile a second time with --strip-extras to generate the constraints file, I ended up with a different version for one of the packages between the generated constraints and requirements. I'm guessing a new version of the package between invocations of pip-compile; generating both files at the same time would have avoided this. (Removing both files and regenerating resulted in matching files).

cout avatar Mar 27 '23 21:03 cout