ggshield icon indicating copy to clipboard operation
ggshield copied to clipboard

ggshield secret scan pypi fails if the Python version is not supported by pip

Open agateau-gg opened this issue 2 years ago • 0 comments

Environment

  • ggshield version: 1.14.2
  • Operating system (Linux, macOS, Windows): Any
  • Operating system version: Any
  • Python version: Any

Describe the bug

ggshield secret scan pypi uses the pip command to download packages to scan. If a package requires, for example, Python >= 3.11 and our version of pip is for Python 3.10, then the scan fails.

Here is an example trying to scan vkpybot, which requires 3.11 with a 3.10 Python.

$ ggshield secret scan pypi vkpybot==0.1.10
Downloading pip package... ERROR: Ignored the following versions that require a different python version: 0.1.10 Requires-Python >=3.11,<4.0; 0.1.11 Requires-Python >=3.11,<4.0; 0.1.3 Requires-Python >=3.11,<4.0; 0.1.4 Requires-Python >=3.11,<4.0; 0.1.5 Requires-Python >=3.11,<4.0; 0.1.6 Requires-Python >=3.11,<4.0; 0.1.7 Requires-Python >=3.11,<4.0; 0.1.8 Requires-Python >=3.11,<4.0
ERROR: Could not find a version that satisfies the requirement vkpybot==0.1.10 (from versions: 0.2.0)
ERROR: No matching distribution found for vkpybot==0.1.10
Error: Failed to download "vkpybot==0.1.10"

Possible solution

Use unearth instead of pip. The example below shows how unearth used with Python 3.8 finds vkpybot 0.1.10 even if it requires Python >=3.11.0:

Python 3.8.17 (default, Jul 28 2023, 06:03:56) 
[GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from unearth import PackageFinder

# `ignore_compatibility=True` is the important bit here
>>> finder = PackageFinder(index_urls=["https://pypi.org/simple/"], ignore_compatibility=True)

>>> match = finder.find_best_match("vkpybot==0.1.10")
>>> match
BestMatch(best=Package(name='vkpybot', version='0.1.10'), applicable=<unearth.utils.LazySequence object at 0x7f980f6c2370>, candidates=<unearth.utils.LazySequence object at 0x7f980f6c24f0>)
>>> package = match.best
>>> package.link.url
'https://files.pythonhosted.org/packages/3e/b9/2cc5f92a14ad9763220b589faa6ed5b8f5afa91bbd3b30e39a2b23270094/vkpybot-0.1.10-py3-none-any.whl'
>>> package.link.requires_python
'>=3.11,<4.0'

This would probably solve #394 as well.

agateau-gg avatar Jan 23 '23 14:01 agateau-gg