pip-tools

pip-tools copied to clipboard

What's the problem this feature will solve?

I have a repo with multiple sub-packages inside subfolders. At the repo root, we have vanilla requirements.txt, and for each sub-package I use pip-compile. Furthermore, I use pre-commit's requirements-txt-fixer hook for the requirements files in the repo root.

One of my sub-packages depends on setuptools, which is considered unsafe. Thus, pip-compile --allow-unsafe puts it in its own section at the bottom of the generated requirements.txt after a newline.

Unfortunately, upon commit the requirements-txt-fixer then auto-sorts setuptools into the main body of the requirements.txt to be alphabetically ordered (and doesn't migrate the # comments).

Describe the solution you'd like

I would like either:

• --allow-unsafe to have unsafe packages be included in the alphabetized main requirements' body
• An additional flag to enable unsafe packages to be included in the alphabetized main requirements' body

So instead of:

#
# This file is autogenerated by pip-compile with python 3.10
# To update, run:
#
#    pip-compile --allow-unsafe --no-emit-index-url requirements.in
#
attrs==21.4.0
    # via pytest
pytest==7.1.2
    # via
    #   -r requirements.in

# The following packages are considered to be unsafe in a requirements file:
setuptools==63.2.0
    # via -r requirements.in

It becomes:

#
# This file is autogenerated by pip-compile with python 3.10
# To update, run:
#
#    pip-compile --allow-unsafe --no-emit-index-url requirements.in
#
attrs==21.4.0
    # via pytest
pytest==7.1.2
    # via
    #   -r requirements.in
setuptools==63.2.0
    # via -r requirements.in
    # This package is considered to be unsafe in a requirements file

Alternative Solutions

Workaround: I can have requirements-txt-fixer only work at the repo root level in my pre-commit config using files regex:

repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.3.0
    hooks:
      - id: requirements-txt-fixer
        description: >-
          Sorts entries in requirements.txt and removes incorrect entry for
          `pkg-resources==0.0.0`.
        files: ^req

However, this means any other requirements files in sub-packages not generated by pip-compile will be untouched by requirements-txt-fixer. I could modify the files regex to be more specific, but that feels one-off-y.

Additional context

I think I found a bug related to --unsafe-package with normalization as adding --unsafe-package MarkupSafe works fine but if you use --unsafe-package markupsafe the outcome is not the desired one. Apparently it does not work with already normalized arguments.

Having the same issue here, we have to use files filter workaround on pre-commit to mitigate the issue