pip-audit icon indicating copy to clipboard operation
pip-audit copied to clipboard

Published 20 hours ago • verified publisher icon pypa

pypi_provider asks for bogus requirement pkg_resources==0.0.0

Open m000 opened this issue 3 years ago • 12 comments

Bug description

When running pip-audit with some requirements files, it fails to complete because bogus requirement pkg_resources==0.0.0 is introduced by pypi_provider. E.g. resolvelib.resolvers.ResolutionImpossible: [RequirementInformation(requirement=<Requirement('pkg_resources==0.0.0')>, parent=<django-admin-inline-paginator==0.2.2 wheel=False>)]

If the requirements are installed in a virtualenv and pip-audit is run from the virtualenv without further arguments, it works as expected.

Reproduction steps

  • Test requirements file: 
    asgiref==3.4.1
Django==3.2.11
django-admin-inline-paginator==0.2.2
sqlparse==0.4.2
  • Doesn't work (see detailed log below): 
    pip-audit --progress-spinner off -r requirements.txt
  • Works: 
    python -m venv pyenv
. ./pyenv/bin/activate
pip install --progress-spinner off -r requirements.txt
pip-audit

Expected behavior

It should work in both cases.

Screenshots and logs

Command output:

root@e6916ccc1c6e:/home/jenkins/requirements/pip-audit# pip-audit --progress-spinner off -r requirements.txt
Processing /tmp/tmps34xwl6x/django-admin-inline-paginator-0.2.2.tar.gz
  Preparing metadata (setup.py) ... done
Collecting django
  Using cached Django-4.0.1-py3-none-any.whl (8.0 MB)
Collecting sqlparse>=0.2.2
  Using cached sqlparse-0.4.2-py3-none-any.whl (42 kB)
Collecting asgiref<4,>=3.4.1
  Using cached asgiref-3.4.1-py3-none-any.whl (25 kB)
Building wheels for collected packages: django-admin-inline-paginator
  Building wheel for django-admin-inline-paginator (setup.py) ... done
  Created wheel for django-admin-inline-paginator: filename=django_admin_inline_paginator-0.2.2-py3-none-any.whl size=8867 sha256=8d31eec5a7c40c2e2e52599e866a0af39a9a3758da17b2783cf4c1cae629f613
  Stored in directory: /root/.cache/pip/wheels/a6/35/82/b063855c1c9dda00d174cf9f7c2152f2bb9690618aa233acf3
Successfully built django-admin-inline-paginator
Installing collected packages: sqlparse, asgiref, django, django-admin-inline-paginator
Successfully installed asgiref-3.4.1 django-4.0.1 django-admin-inline-paginator-0.2.2 sqlparse-0.4.2
Traceback (most recent call last):
  File "/home/jenkins/requirements/pip-audit/pyenv/bin/pip-audit", line 8, in <module>
    sys.exit(audit())
  File "/home/jenkins/requirements/pip-audit/pyenv/lib/python3.9/site-packages/pip_audit/_cli.py", line 268, in audit
    for (spec, vulns) in auditor.audit(source):
  File "/home/jenkins/requirements/pip-audit/pyenv/lib/python3.9/site-packages/pip_audit/_audit.py", line 60, in audit
    yield from self._service.query_all(specs)
  File "/home/jenkins/requirements/pip-audit/pyenv/lib/python3.9/site-packages/pip_audit/_service/interface.py", line 115, in query_all
    for spec in specs:
  File "/home/jenkins/requirements/pip-audit/pyenv/lib/python3.9/site-packages/pip_audit/_dependency_source/requirement.py", line 63, in collect
    for _, deps in self.resolver.resolve_all(iter(req_values)):
  File "/home/jenkins/requirements/pip-audit/pyenv/lib/python3.9/site-packages/pip_audit/_dependency_source/interface.py", line 67, in resolve_all
    yield (req, self.resolve(req))
  File "/home/jenkins/requirements/pip-audit/pyenv/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/resolvelib.py", line 53, in resolve
    result = self.resolver.resolve([req])
  File "/home/jenkins/requirements/pip-audit/pyenv/lib/python3.9/site-packages/resolvelib/resolvers.py", line 481, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/home/jenkins/requirements/pip-audit/pyenv/lib/python3.9/site-packages/resolvelib/resolvers.py", line 385, in resolve
    raise ResolutionImpossible(self.state.backtrack_causes)
resolvelib.resolvers.ResolutionImpossible: [RequirementInformation(requirement=<Requirement('pkg_resources==0.0.0')>, parent=<django-admin-inline-paginator==0.2.2 wheel=False>)]

Verbose output:

root@e6916ccc1c6e:/home/jenkins/requirements/pip-audit# pip-audit --verbose --progress-spinner off -r requirements.txt
DEBUG:pip_audit._cli:parsed arguments: Namespace(local=False, requirements=[<_io.TextIOWrapper name='requirements.txt' mode='r' encoding='UTF-8'>], format=<OutputFormatChoice.Columns: 'columns'>, vulnerability_service=<VulnerabilityServiceChoice.Pypi: 'pypi'>, dry_run=False, strict=False, desc=<VulnerabilityDescriptionChoice.Auto: 'auto'>, cache_dir=None, progress_spinner=<ProgressSpinnerChoice.Off: 'off'>, timeout=15, paths=[], verbose=True)
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/asgiref" in the cache
DEBUG:cachecontrol.controller:Returning cached permanent redirect response (ignoring date and etag information)
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/asgiref/" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 61
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:600 > 61
DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/fe/66/577f32b54c50dcd8dec38447258e82ed327ecb86820d67ae7b3dea784f13/asgiref-3.4.1-py3-none-any.whl" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 22869
DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 365000000
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:365000000 > 22869
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/pypi/asgiref/3.4.1/json" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 66
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 900
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:900 > 66
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/django" in the cache
DEBUG:cachecontrol.controller:Returning cached permanent redirect response (ignoring date and etag information)
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/django/" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 62
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:600 > 62
DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/03/40/1ec2b4abb0c91f0c6195692a9f7a3709f1c0fe95258f3e4d8aa7d8dab92b/Django-3.2.11-py3-none-any.whl" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 22364
DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 365000000
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:365000000 > 22364
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/asgiref" in the cache
DEBUG:cachecontrol.controller:Returning cached permanent redirect response (ignoring date and etag information)
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/asgiref/" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 61
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:600 > 61
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/pytz" in the cache
DEBUG:cachecontrol.controller:Returning cached permanent redirect response (ignoring date and etag information)
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/pytz/" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 195
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:600 > 195
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/sqlparse" in the cache
DEBUG:cachecontrol.controller:Returning cached permanent redirect response (ignoring date and etag information)
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/sqlparse/" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 62
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:600 > 62
DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/fe/66/577f32b54c50dcd8dec38447258e82ed327ecb86820d67ae7b3dea784f13/asgiref-3.4.1-py3-none-any.whl" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 22869
DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 365000000
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:365000000 > 22869
DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/05/40/d836d55fb3f467243ee839ab7b814822fda522cd395fa41e282684e71ee5/sqlparse-0.4.2-py3-none-any.whl" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 22866
DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 365000000
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:365000000 > 22866
DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/d3/e3/d9f046b5d1c94a3aeab15f1f867aa414f8ee9d196fae6865f1d6a0ee1a0b/pytz-2021.3-py2.py3-none-any.whl" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 22579
DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 365000000
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:365000000 > 22579
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/pypi/django/3.2.11/json" in the cache
DEBUG:cachecontrol.controller:Returning cached permanent redirect response (ignoring date and etag information)
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/pypi/Django/3.2.11/json" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 65
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 900
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:900 > 65
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/pypi/sqlparse/0.4.2/json" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 65
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 900
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:900 > 65
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/pypi/pytz/2021.3/json" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 65
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 900
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:900 > 65
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/django-admin-inline-paginator" in the cache
DEBUG:cachecontrol.controller:Returning cached permanent redirect response (ignoring date and etag information)
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/django-admin-inline-paginator/" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 195
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:600 > 195
DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/4c/b5/d3abb8682b168cb2d4119879215428ca63b33b524a6f1f4dc6f11ef14dcf/django-admin-inline-paginator-0.2.2.tar.gz" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 22548
DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 365000000
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:365000000 > 22548
Processing /tmp/tmpecurcpka/django-admin-inline-paginator-0.2.2.tar.gz
  Preparing metadata (setup.py) ... done
Collecting django
  Using cached Django-4.0.1-py3-none-any.whl (8.0 MB)
Collecting sqlparse>=0.2.2
  Using cached sqlparse-0.4.2-py3-none-any.whl (42 kB)
Collecting asgiref<4,>=3.4.1
  Using cached asgiref-3.4.1-py3-none-any.whl (25 kB)
Building wheels for collected packages: django-admin-inline-paginator
  Building wheel for django-admin-inline-paginator (setup.py) ... done
  Created wheel for django-admin-inline-paginator: filename=django_admin_inline_paginator-0.2.2-py3-none-any.whl size=8867 sha256=286b8aeb24e9153695108c12919029209828fd51b701693b272002e833229766
  Stored in directory: /root/.cache/pip/wheels/22/23/ae/274c6b4a0e54de476df4e41550a01f2dabd996a8d114c143e7
Successfully built django-admin-inline-paginator
Installing collected packages: sqlparse, asgiref, django, django-admin-inline-paginator
Successfully installed asgiref-3.4.1 django-4.0.1 django-admin-inline-paginator-0.2.2 sqlparse-0.4.2
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/asgiref" in the cache
DEBUG:cachecontrol.controller:Returning cached permanent redirect response (ignoring date and etag information)
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/asgiref/" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 2
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:600 > 2
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/django" in the cache
DEBUG:cachecontrol.controller:Returning cached permanent redirect response (ignoring date and etag information)
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/django/" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 3
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:600 > 3
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/django-admin-inline-paginator" in the cache
DEBUG:cachecontrol.controller:Returning cached permanent redirect response (ignoring date and etag information)
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/django-admin-inline-paginator/" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 201
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:600 > 201
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/pip" in the cache
DEBUG:cachecontrol.controller:Returning cached permanent redirect response (ignoring date and etag information)
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/pip/" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 2
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:600 > 2
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/pkg-resources" in the cache
DEBUG:cachecontrol.controller:Returning cached permanent redirect response (ignoring date and etag information)
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/pkg-resources/" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 194
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:600 > 194
Traceback (most recent call last):
  File "/home/jenkins/requirements/pip-audit/pyenv/bin/pip-audit", line 8, in <module>
    sys.exit(audit())
  File "/home/jenkins/requirements/pip-audit/pyenv/lib/python3.9/site-packages/pip_audit/_cli.py", line 268, in audit
    for (spec, vulns) in auditor.audit(source):
  File "/home/jenkins/requirements/pip-audit/pyenv/lib/python3.9/site-packages/pip_audit/_audit.py", line 60, in audit
    yield from self._service.query_all(specs)
  File "/home/jenkins/requirements/pip-audit/pyenv/lib/python3.9/site-packages/pip_audit/_service/interface.py", line 115, in query_all
    for spec in specs:
  File "/home/jenkins/requirements/pip-audit/pyenv/lib/python3.9/site-packages/pip_audit/_dependency_source/requirement.py", line 63, in collect
    for _, deps in self.resolver.resolve_all(iter(req_values)):
  File "/home/jenkins/requirements/pip-audit/pyenv/lib/python3.9/site-packages/pip_audit/_dependency_source/interface.py", line 67, in resolve_all
    yield (req, self.resolve(req))
  File "/home/jenkins/requirements/pip-audit/pyenv/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/resolvelib.py", line 53, in resolve
    result = self.resolver.resolve([req])
  File "/home/jenkins/requirements/pip-audit/pyenv/lib/python3.9/site-packages/resolvelib/resolvers.py", line 481, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/home/jenkins/requirements/pip-audit/pyenv/lib/python3.9/site-packages/resolvelib/resolvers.py", line 385, in resolve
    raise ResolutionImpossible(self.state.backtrack_causes)
resolvelib.resolvers.ResolutionImpossible: [RequirementInformation(requirement=<Requirement('pkg_resources==0.0.0')>, parent=<django-admin-inline-paginator==0.2.2 wheel=False>)]

Platform information

  • OS name and version: Debian 11, Docker
  • pip-audit version (pip-audit -V): pip-audit 1.1.1
  • Python version (python -V or python3 -V): Python 3.9.2
  • pip version (pip -V or pip3 -V): pip 21.3.1 from /root/.local/lib/python3.9/site-packages/pip (python 3.9)

Additional context

The exception trace is in resolvelib. But I've tracked down where the bogus dependency is generated. It boils down to this function in resolvers.py. Which from what I understand is implemented by pypi_provider.py of pip-audit.

def _get_updated_criteria(self, candidate):
      criteria = self.state.criteria.copy()
      for requirement in self._p.get_dependencies(candidate=candidate):
          self._add_to_criteria(criteria, requirement, parent=candidate)
      return criteria

m000 avatar Jan 06 '22 23:01 m000

This is related to https://bugs.debian.org/994952: Debian has a patch to virtualenv to explicitly install pkg_resources (as it's a separate binary package and wheel, from setuptools, in Debian), but it doesn't exist on PyPI. This doesn't affect you when using python -m venv because this isn't using the patched virtualenv.

I suspect specifying the -l/--local flag to pip-audit might fix this for you.

di avatar Jan 06 '22 23:01 di

Dates back to at least 2018 as well: https://stackoverflow.com/questions/38992194/why-does-pip-freeze-list-pkg-resources-0-0-0

I'm not sure there's much pip-audit can do here: we're faithfully retrieving the metadata that the environment gives us, which for some Linux distros is just incorrect and can't be reconciled.

woodruffw avatar Jan 06 '22 23:01 woodruffw

@di Yes, -l sounds promising, but it made no difference.

@woodruffw Seems relevant, but not exactly. I don't get pkg-resources in my pip listing, only setuptools:

$ pip freeze --all
pip==20.3.4
setuptools==52.0.0
wheel==0.34.2

I was wondering, are there any other packages that use 0.0.0 as version? This seems like a legit heuristic for excluding a package from the dependencies list.

Other than that, adding some sort of --ignore option could be a workaround. This could also come handy in other situations. E.g. run pip-audit as part of your CI/CD pipeline, but don't fail for packages that you know that have vulnerabilities, but you can't upgrade for one reason or another.

I can prob implement either of these and send a PR if needed.

m000 avatar Jan 07 '22 00:01 m000

PEP 440 says the following:

All numeric components MAY be zero. Except as described below for the release segment, a numeric component of zero has no special significance aside from always being the lowest possible value in the version ordering.

...which unfortunately means that publishing a 0.0.0 distribution on PyPI is perfectly valid, and dependency resolvers must respect and handle it correctly. So that heuristic won't be safe, unfortunately.

Other than that, adding some sort of --ignore option could be a workaround. This could also come handy in other situations. E.g. run pip-audit as part of your CI/CD pipeline, but don't fail for packages that you know that have vulnerabilities, but you can't upgrade for one reason or another.

Yep! #209 also raises this idea; I think it's something we want, but we'll want to flesh out a reasonable design first. In particular, allowing users to ignore arbitrary packages has the potential to make pip-audit -r perform even more unreliably than it currently does, so we'll want to be careful about how we allow ignoring.

woodruffw avatar Jan 07 '22 00:01 woodruffw

pkg-resources is a weird case. It has an empty link list page (HTTP 200) but no project page (HTTP 404).

We want to switch from Safety to pip-audit, so this was really a show-stopper. I have an incoming PR that adds a flag to the cli for skipping unresolvable packages. Debian is an important distribution, so it would probably be for the benefit of the project to have a workaround for this quirk. But if you prefer to wait for a better solution, it can happily live in my fork.

The PR could perhaps be a bit more specialized by having get_project_from_pypi() raise a PyPINotFoundError or a (new) PyPINoProjectLinksError if it can find no links in the links page.

m000 avatar Jan 07 '22 18:01 m000

pkg-resources is a weird case. It has an empty link list page (HTTP 200) but no project page (HTTP 404).

Yeah, it's strange that it has a simple index page at all -- pkg-resources is a wholly-contained sub-package of setuptools, so it really shouldn't be independently listed/indexed at all.

Thanks for making the PR! I'll take a look now.

woodruffw avatar Jan 07 '22 18:01 woodruffw

Let me know what you think. Also looking to #197, it is probably better to create a new exception so we can tell different case of failed resolution apart.

m000 avatar Jan 07 '22 18:01 m000

Just for future visiblity: here are the xrefs for the same pkg_resources=0.0.0 pain in pip: https://github.com/pypa/pip/issues/8331 https://github.com/pypa/pip/issues/4022

woodruffw avatar Jan 07 '22 18:01 woodruffw

Just looping back on this: if you have a fully resolved requirements.txt, passing --no-deps might avoid this error for you (since it won't attempt to do any extra resolution).

woodruffw avatar May 26 '22 21:05 woodruffw

Running pip-audit on Ubuntu seems to break lots of things which work fine in other environments (I am developing on Gentoo but with an Ubuntu CI environment). What works in Ubuntu 18.04 and 20.04 is as follows:

  • Don't try to install pip-audit in the Ubuntu environment.
  • Create an empty virtual environment
  • Install just pip-audit in the environment
  • Audit your requirements file from within the (nearly empty) virtual environment.
python3 -m venv .venv
source .venv/bin/activate
pip install pip-audit
pip-audit --strict -r requirements.txt

Inserting the extra layer between the OS and pip-audit seems to be necessary to get rid of interference from the Ubuntu packaging (not just this issue, but also a bunch of packages with no upstream on pypi etc.).

richardash1981 avatar Jun 01 '22 18:06 richardash1981

Yeah, virtual environments are strongly recommended in general, but particularly on Ubuntu/Debian, where the upstream maintainers continue to debundle core parts of Python.

Assuming that you're following the other best practices for Python packaging and deployment, your project should already have its own virtual environment. If that's the case, then you can do this instead:

(env) $ python -m pip install pip-audit
(env) $ pip-audit

...and that'll automatically audit the current (virtual) environment, which will also be much faster than having to re-do resolution in requirements mode (-r requirements.txt).

woodruffw avatar Jun 01 '22 19:06 woodruffw

Note that https://peps.python.org/pep-0668/ might eventually help with this, but it'll probably be a while.

di avatar Jun 01 '22 20:06 di