ggshield
ggshield copied to clipboard
ggshield secret scan pypi fails if the Python version is not supported by pip
Environment
- ggshield version: 1.14.2
- Operating system (Linux, macOS, Windows): Any
- Operating system version: Any
- Python version: Any
Describe the bug
ggshield secret scan pypi
uses the pip
command to download packages to scan. If a package requires, for example, Python >= 3.11 and our version of pip
is for Python 3.10, then the scan fails.
Here is an example trying to scan vkpybot, which requires 3.11 with a 3.10 Python.
$ ggshield secret scan pypi vkpybot==0.1.10
Downloading pip package... ERROR: Ignored the following versions that require a different python version: 0.1.10 Requires-Python >=3.11,<4.0; 0.1.11 Requires-Python >=3.11,<4.0; 0.1.3 Requires-Python >=3.11,<4.0; 0.1.4 Requires-Python >=3.11,<4.0; 0.1.5 Requires-Python >=3.11,<4.0; 0.1.6 Requires-Python >=3.11,<4.0; 0.1.7 Requires-Python >=3.11,<4.0; 0.1.8 Requires-Python >=3.11,<4.0
ERROR: Could not find a version that satisfies the requirement vkpybot==0.1.10 (from versions: 0.2.0)
ERROR: No matching distribution found for vkpybot==0.1.10
Error: Failed to download "vkpybot==0.1.10"
Possible solution
Use unearth instead of pip. The example below shows how unearth used with Python 3.8 finds vkpybot 0.1.10 even if it requires Python >=3.11.0:
Python 3.8.17 (default, Jul 28 2023, 06:03:56)
[GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from unearth import PackageFinder
# `ignore_compatibility=True` is the important bit here
>>> finder = PackageFinder(index_urls=["https://pypi.org/simple/"], ignore_compatibility=True)
>>> match = finder.find_best_match("vkpybot==0.1.10")
>>> match
BestMatch(best=Package(name='vkpybot', version='0.1.10'), applicable=<unearth.utils.LazySequence object at 0x7f980f6c2370>, candidates=<unearth.utils.LazySequence object at 0x7f980f6c24f0>)
>>> package = match.best
>>> package.link.url
'https://files.pythonhosted.org/packages/3e/b9/2cc5f92a14ad9763220b589faa6ed5b8f5afa91bbd3b30e39a2b23270094/vkpybot-0.1.10-py3-none-any.whl'
>>> package.link.requires_python
'>=3.11,<4.0'
This would probably solve #394 as well.