poetry
poetry copied to clipboard
python-poetry
Poetry is incompatible with unstable-tagged Python builds (Invalid PEP 440 version: '3.Y.Z+')
I'v just installed Python3.11 from deadsnakes/nightly.
Poetry can't use it, because the Python version has a + in it.
Invalid PEP 440 version: '3.11.0+' is all I get 😭
P.S. if the wise men declare that deadsnakes is at fault here, I'll happily bug dead snakes.
Hmm, it looks like this could be a parse error with the Python version numbers provided by Deadsnakes? Looking closer at #6334, it looks like this might be a dupe. That issue was closed as a environment issue caused by bad versions in the target environment, but looking closer, that only represents the comments. The issue body, like this one, seems to be about a Python version.
Deadsnakes versions tend to be like 3.11.0+jammy1, which should be valid; some investigation of the actual version numbers from sys would be quite helpful. We might have to add some logic to strip extra tokens from Python versions in any case -- they really don't make sense for interpreter version comparisons as we don't support anything other than simple three-part-tuple versions anyway.
>>> import sys
>>> sys.version
'3.11.0+ (main, Oct 30 2022, 08:56:33) [GCC 9.4.0]'
>>> sys.version_info
sys.version_info(major=3, minor=11, micro=0, releaselevel='final', serial=0)
In the stack trace you can see that python_full_version is the culprit. According to PEP 508 python_full_version is equal to platform.python_version(), which is 3.11+ in this case. Although I could not find a direct reference, the sample values for python_full_version in PEP 508 indicate that it should be PEP 440 compliant.
Stack trace
Finding the necessary packages for the current system
Stack trace:
16 ~/.local/pipx/venvs/poetry/lib/python3.8/site-packages/cleo/application.py:329 in run
exit_code = self._run(io)
15 ~/.local/pipx/venvs/poetry/lib/python3.8/site-packages/poetry/console/application.py:185 in _run
exit_code: int = super()._run(io)
14 ~/.local/pipx/venvs/poetry/lib/python3.8/site-packages/cleo/application.py:423 in _run
exit_code = self._run_command(command, io)
13 ~/.local/pipx/venvs/poetry/lib/python3.8/site-packages/cleo/application.py:465 in _run_command
raise error
12 ~/.local/pipx/venvs/poetry/lib/python3.8/site-packages/cleo/application.py:449 in _run_command
exit_code = command.run(io)
11 ~/.local/pipx/venvs/poetry/lib/python3.8/site-packages/cleo/commands/base_command.py:119 in run
status_code = self.execute(io)
10 ~/.local/pipx/venvs/poetry/lib/python3.8/site-packages/cleo/commands/command.py:83 in execute
return self.handle()
9 ~/.local/pipx/venvs/poetry/lib/python3.8/site-packages/poetry/console/commands/install.py:146 in handle
return_code = self.installer.run()
8 ~/.local/pipx/venvs/poetry/lib/python3.8/site-packages/poetry/installation/installer.py:115 in run
return self._do_install()
7 ~/.local/pipx/venvs/poetry/lib/python3.8/site-packages/poetry/installation/installer.py:320 in _do_install
with solver.use_environment(self._env):
6 /usr/lib/python3.8/contextlib.py:113 in __enter__
return next(self.gen)
5 ~/.local/pipx/venvs/poetry/lib/python3.8/site-packages/poetry/puzzle/solver.py:65 in use_environment
with self.provider.use_environment(env):
4 /usr/lib/python3.8/contextlib.py:113 in __enter__
return next(self.gen)
3 ~/.local/pipx/venvs/poetry/lib/python3.8/site-packages/poetry/puzzle/provider.py:174 in use_environment
self._python_constraint = Version.parse(env.marker_env["python_full_version"])
2 ~/.local/pipx/venvs/poetry/lib/python3.8/site-packages/poetry/core/version/pep440/version.py:182 in parse
return parse_pep440(value, cls)
1 ~/.local/pipx/venvs/poetry/lib/python3.8/site-packages/poetry/core/version/pep440/parser.py:83 in parse_pep440
return PEP440Parser.parse(value, version_class)
InvalidVersion
Invalid PEP 440 version: '3.11.0+'
at ~/.local/pipx/venvs/poetry/lib/python3.8/site-packages/poetry/core/version/pep440/parser.py:69 in parse
65│ @classmethod
66│ def parse(cls, value: str, version_class: type[T]) -> T:
67│ match = cls._regex.search(value) if value else None
68│ if not match:
→ 69│ raise InvalidVersion(f"Invalid PEP 440 version: '{value}'")
70│
71│ return version_class(
72│ epoch=int(match.group("epoch")) if match.group("epoch") else 0,
73│ release=cls._get_release(match),
I re-read PEP 508 and there seems to be an escape hatch for non-compliant versions:
The <version_cmp> operators use the PEP 440 version comparison rules when those are defined (that is when both sides have a valid version specifier). If there is no defined PEP 440 behaviour and the operator exists in Python, then the operator falls back to the Python behaviour.
However, that implies python_full_version >= "3.9" is fulfilled for Python 3.11, but not for deadsnakes Python 3.11+. (IIUC, a version comparison should be used for the first one and a string comparison for the second one.) That can't be the intention of deadsnakes authors!? Maybe, I miss something but it would probably be better to use PEP 440 compliant versions. Even if poetry would support these weird versions, the consequences would be unexpected for most users. 🤔
Appending a '+' seems to be the convention in the python source
https://github.com/python/cpython/blob/a9a8c8712665377cfa83af4b632b0db529ec1853/Include/patchlevel.h#L26 is presumably the origin of this
you could try persuading them that using a PEP440-compatible version would be more sensible
https://github.com/python/cpython/issues/88166
looks like that became https://github.com/python-poetry/poetry/issues/4021, but the reporter closed it just twenty minutes after opening it...
I have the same issue using python 3.11 packaged inside the official debian repositories. My python3.11 --version output is Python 3.11.0+
Btw., deadsnakes Python 3.10 also suffers this fate.
My current workaround: sudo hexedit /usr/bin/python3.10 find the version and overwrite the + with a zero byte.
And same for 3.11.
🙏🏻 project maintainers, please consider this important 🙏🏻
I think it's obvious that deadsnakes in general is broken with Poetry as both 3.11 and 3.10 are GA versions.
more like: choosing to use the nightly builds for python 3.10 and python 3.11 is the wrong choice for almost everyone at this point, since there are stable releases available.
@dimaqq, I think you misunderstood my meaning -- if 3.11 is broken, it is reasonably obvious that 3.10 is also broken, as they are both GA versions (so the + is not a transitive property of a pre-release build).
the regular 3.10 build is not 'broken' and I imagine neither is 3.11, rather folk are choosing to use nightly builds.
I get my 3.10 from deadsnakes myself, and the versioning is completely normal:
$ cat /etc/apt/sources.list.d/deadsnakes-ubuntu-ppa-focal.list
deb http://ppa.launchpad.net/deadsnakes/ppa/ubuntu focal main
# deb-src http://ppa.launchpad.net/deadsnakes/ppa/ubuntu focal main
$ python3.10 --version
Python 3.10.8
$ python3.10 -c 'import platform ; print(platform.python_version())'
3.10.8
that still leaves poetry confused by version numbers from nightly builds: but nearly no-one should have any reason to use those, certainly for pythons 3.10 and 3.11
Ah, that would have been nice to have included in the reproduction and adds important context; the severity of this is much lower than I believed based on that.
to be fair it is the very first sentence of the very opening of this thread
The official package from the current Debian Testing and Unstable (https://tracker.debian.org/pkg/python3.11) has this issue, and it is not using nightly builds.
❯ which python3.11
/usr/bin/python3.11
❯ dpkg -S /usr/bin/python3.11
python3.11-minimal: /usr/bin/python3.11
❯ python3.11 --version
Python 3.11.0+
um, it obviously is using nightly builds. You can tell by the version number!
probably "testing and unstable" is the clue
Indeed, I am not worried about that case as much:
-
python3.11is explicitly experimental in unstable/testing; Debian does not ship Python versions in parallel under normal circumstances - Unstable/testing are moving targets and not reasonable environments for 95% of users; those who are using them are theoretically equipped to understand and mitigate this kind of transient issue
- Debian Python packaging is always idiosyncratic and bordering on broken; there is a maintainer who is invested in papering over decisions made by Debian, but he has had little time lately and no one else is invested in it
um, it obviously is using nightly builds. You can tell by the version number!
Just checked and you are right. It is not using the actual final but some other commit from the 3.11 branch. Sorry about assuming it was, the packages from debian usually use official released tarballs.
probably "testing and unstable" is the clue
Not really. The name is misleading, but testing is just an alias to the codename that will become the next stable version, and unstable is where packages get uploaded and automatically get migrated to the testing one if not issues are found. The debian team doesn't usually upload nightly versions to any of those repositories (there's a experimental repository for that there), and those packages will be the ones in the final stable release (just as they are, no repacking or recompiling) as long as no critical issues are found and no new versions are released and packaged.
Debian Python person here:
python3.11 is explicitly experimental in unstable/testing; Debian does not ship Python versions in parallel under normal circumstances
It's in the process of becoming supported. Hopefully for the next stable release, but that depends on how much of the world works with it, and whether we can fix the rest in time.
Unstable/testing are moving targets and not reasonable environments for 95% of users; those who are using them are theoretically equipped to understand and mitigate this kind of transient issue Debian Python packaging is always idiosyncratic and bordering on broken; there is a maintainer who is invested in papering over decisions made by Debian, but he has had little time lately and no one else is invested in it
I wouldn't really agree with either of these sentiments. But bugs aren't the place to litigate that. I really don't think Debian did anything wrong here. Poetry didn't expect to see a + in a Python version, but they are now something to be expected, since 3.11. That's about the extent of the issue, here.
It's in the process of becoming supported. Hopefully for the next stable release, but that depends on how much of the world works with it, and whether we can fix the rest in time.
Right, but 3.10 is currently the primary Python version, and will be removed if there is a cut-over to 3.11, right?
but they are now something to be expected, since 3.11. That's about the extent of the issue, here.
Is that a statement of intent by Debian? The + signifies it's not a final build as tagged by upstream -- does Debian intend to ship a 3.11 based on an untagged commit?
Right, but 3.10 is currently the primary Python version, and will be removed if there is a cut-over to 3.11, right?
Yes, it'll be removed when possible.
Is that a statement of intent by Debian? The + signifies it's not a final build as tagged by upstream -- does Debian intend to ship a 3.11 based on an untagged commit?
We regularly ship a patch bringing in all the latest commits in the maintenance branch, not just the tagged releases. That's not my decision, so I don't know the full thinking there, but we often need those patches, because they resolve issues we're facing.
but they are now something to be expected, since 3.11
To be clear, I was saying this is now something to be seen, in general. Not specifically in Debian. It's common to be working with Python built out of the source tree (when developing cpython itself, or chasing down bugs in it, for example).
I don't think Poetry should restrict itself to functioning correctly on release tags of cPython.
I don't think Poetry should restrict itself to functioning correctly on release tags of cPython.
I don't think anyone disputes that, but this is certainly not as hot a fire as we originally thought. PRs are welcome but this will likely be moderately invasive/painful to fix.
Hi, what about something like this? (it works in my local tests in Debian)
Index: src/poetry/utils/env.py
===================================================================
--- src/poetry/utils/env.py 2022-11-30 20:30:11.425524136 -0300
+++ src/poetry/utils/env.py 2022-11-30 20:30:31.829803450 -0300
@@ -1618,7 +1618,7 @@
"platform_release": platform.release(),
"platform_system": platform.system(),
"platform_version": platform.version(),
- "python_full_version": platform.python_version(),
+ "python_full_version": platform.python_version().rstrip("+"),
"platform_python_implementation": platform.python_implementation(),
"python_version": ".".join(platform.python_version().split(".")[:2]),
"sys_platform": sys.platform,
That's far from the only place where we request the full Python version, and we'll still need to create robust tests. It's also very painfully hardcoded and without explanation outside blame; ideally we solve this in the code that actually parses the version.
I wouldn't strip the "+" in general in the code parsing versions because it's definitively not valid for package versions (and at least a bad choice for the python version). I haven't searched for other places yet but remembering the failure on 3.11+ I think that is at least part of a correct fix. Of course we should add a comment why we are stripping the "+" and a test would be nice.
My thinking is we should centralize it using a parse_python_version() variant that encapsulates the stripping of the + so we don't randomly proliferate it in the codebase and get burned by a future omission. The parse_python_version() code can just be a preprocessing step before we hand it to our regular version parsing code, and can be unit tested; we should also end-to-end test the main usage (in env.py).
Not sure whether it brings anything useful to the table, but the Python 3.12.0a2 'release' (installed using pyenv) also reports its version as 3.12.0a2+ and thus fails to work with Poetry.
BTW, a simple workaround is to first create the virtualenv using the venv package (from the Python version you want to use), which Poetry will happily use.