poetry
poetry copied to clipboard
python-poetry
_add_incompatibility AssertionError with prodigy wheel
- Poetry version: 1.2.1
- Python version: 3.9.9
- OS version and name: macOS 12.6 (M1)
- pyproject.toml:
[tool.poetry]
name = "prodigy"
version = "0.1.0"
description = ""
authors = ["Nicolai Bjerre Pedersen <[email protected]>"]
readme = "README.md"
[tool.poetry.dependencies]
python = "^3.9"
[[tool.poetry.source]]
name = "prodigy"
url = "https://download.prodi.gy"
default = false
secondary = false
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
- [x] I am on the latest stable Poetry version, installed using a recommended method.
- [x] I have searched the issues of this repo and believe that this is not a duplicate.
- [x] I have consulted the FAQ and blog for any relevant entries or release notes.
- [x] If an exception occurs when executing a command, I executed it again in debug mode (
-vvvoption) and have included the output below.
Issue
I'm trying to install a package from a private repo but I'm seeing different bugs in the process. First I add the repo by
poetry source add prodigy "https://download.prodi.gy"
Then I add http-basic auth with
poetry config http-basic.prodigy <my-licence-key> ""
And finally I'm trying to add the package with
poetry add --source=prodigy prodigy -vvv
which yields
Loading configuration file /Users/nix/Library/Preferences/pypoetry/config.toml
Loading configuration file /Users/nix/Library/Preferences/pypoetry/auth.toml
Adding repository prodigy (https://download.prodi.gy)
Using virtualenv: /Users/nix/Projects/rnd/prodigy/.venv
Project environment contains an empty path in sys_path, ignoring.
[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
Creating new session for download.prodi.gy
[urllib3.connectionpool] Starting new HTTPS connection (1): download.prodi.gy:443
[urllib3.connectionpool] https://download.prodi.gy:443 "GET /prodigy/ HTTP/1.1" 307 None
[urllib3.connectionpool] https://download.prodi.gy:443 "GET /prodigy HTTP/1.1" 200 716848
Source (prodigy): Response URL https://download.prodi.gy/prodigy differs from request URL https://download.prodi.gy/prodigy/
Source (prodigy): 49 packages found for prodigy *
Using version ^1.11.8 for prodigy
Updating dependencies
Resolving dependencies...
1: fact: prodigy is 0.1.0
1: derived: prodigy
1: Version solving took 0.002 seconds.
1: Tried 1 solutions.
Stack trace:
16 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:329 in run
327│
328│ try:
→ 329│ exit_code = self._run(io)
330│ except Exception as e:
331│ if not self._catch_exceptions:
15 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/poetry/console/application.py:185 in _run
183│ self._load_plugins(io)
184│
→ 185│ exit_code: int = super()._run(io)
186│ return exit_code
187│
14 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:423 in _run
421│ io.input.set_stream(stream)
422│
→ 423│ exit_code = self._run_command(command, io)
424│ self._running_command = None
425│
13 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:465 in _run_command
463│
464│ if error is not None:
→ 465│ raise error
466│
467│ return event.exit_code
12 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:449 in _run_command
447│
448│ if event.command_should_run():
→ 449│ exit_code = command.run(io)
450│ else:
451│ exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED
11 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/cleo/commands/base_command.py:119 in run
117│ io.input.validate()
118│
→ 119│ status_code = self.execute(io)
120│
121│ if status_code is None:
10 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/cleo/commands/command.py:83 in execute
81│
82│ try:
→ 83│ return self.handle()
84│ except KeyboardInterrupt:
85│ return 1
9 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/poetry/console/commands/add.py:255 in handle
253│ self.installer.whitelist([r["name"] for r in requirements])
254│
→ 255│ status = self.installer.run()
256│
257│ if status == 0 and not self.option("dry-run"):
8 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/poetry/installation/installer.py:114 in run
112│ self._execute_operations = False
113│
→ 114│ return self._do_install()
115│
116│ def dry_run(self, dry_run: bool = True) -> Installer:
7 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/poetry/installation/installer.py:247 in _do_install
245│ source_root=self._env.path.joinpath("src")
246│ ):
→ 247│ ops = solver.solve(use_latest=self._whitelist).calculate_operations()
248│ else:
249│ self._io.write_line("Installing dependencies from lock file")
6 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/poetry/puzzle/solver.py:73 in solve
71│ with self._provider.progress():
72│ start = time.time()
→ 73│ packages, depths = self._solve(use_latest=use_latest)
74│ end = time.time()
75│
5 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/poetry/puzzle/solver.py:151 in _solve
149│
150│ try:
→ 151│ result = resolve_version(
152│ self._package, self._provider, locked=locked, use_latest=use_latest
153│ )
4 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/poetry/mixology/__init__.py:24 in resolve_version
22│ solver = VersionSolver(root, provider, locked=locked, use_latest=use_latest)
23│
→ 24│ return solver.solve()
25│
3 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/poetry/mixology/version_solver.py:127 in solve
125│ while next is not None:
126│ self._propagate(next)
→ 127│ next = self._choose_package_version()
128│
129│ return self._result()
2 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/poetry/mixology/version_solver.py:450 in _choose_package_version
448│ conflict = False
449│ for incompatibility in self._provider.incompatibilities_for(package):
→ 450│ self._add_incompatibility(incompatibility)
451│
452│ # If an incompatibility is already satisfied, then selecting version
1 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/poetry/mixology/version_solver.py:486 in _add_incompatibility
484│
485│ def _add_incompatibility(self, incompatibility: Incompatibility) -> None:
→ 486│ self._log(f"fact: {incompatibility}")
487│
488│ for term in incompatibility.terms:
AssertionError
at ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/poetry/mixology/incompatibility.py:114 in __str__
110│ )
111│
112│ def __str__(self) -> str:
113│ if isinstance(self._cause, DependencyCause):
→ 114│ assert len(self._terms) == 2
115│
116│ depender = self._terms[0]
117│ dependee = self._terms[1]
118│ assert depender.is_positive()
However the first time I tried I got another error: 403 Client Error: Forbidden for url: https://s3.eu-... so I figured that my licence key was wrong (I later realized that it is most likely related to #4497). So I tried to add it again with poetry config http-basic.prodigy <my-licence-key> "" which then gave this error
Unable to store the password for poetry-repository-prodigy in the key ring: Can't store password on keychain: (-25299, 'Unknown Error')
I checked in auth.toml and it looks fine.... If I add verbose logging on the poetry config command then I get
Loading configuration file /Users/nix/Library/Preferences/pypoetry/config.toml
Loading configuration file /Users/nix/Library/Preferences/pypoetry/auth.toml
Adding repository prodigy (https://download.prodi.gy)
[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
Stack trace:
2 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/keyring/backends/macOS/__init__.py:40 in set_password
38│
39│ try:
→ 40│ api.set_generic_password(self.keychain, service, username, password)
41│ except api.KeychainDenied as e:
42│ raise KeyringLocked("Can't store password on keychain: " "{}".format(e))
1 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/keyring/backends/macOS/api.py:161 in set_generic_password
159│
160│ status = SecItemAdd(q, None)
→ 161│ Error.raise_for_status(status)
162│
163│
Error
(-25299, 'Unknown Error')
at ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/keyring/backends/macOS/api.py:114 in raise_for_status
110│ status,
111│ "Security Auth Failure: make sure "
112│ "python is signed with codesign util",
113│ )
→ 114│ raise cls(status, "Unknown Error")
115│
116│
117│ class NotFound(Error):
118│ pass
The following error occurred when trying to handle this error:
Stack trace:
2 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/password_manager.py:85 in set_password
83│
84│ try:
→ 85│ keyring.set_password(name, username, password)
86│ except (RuntimeError, keyring.errors.KeyringError) as e:
87│ raise PoetryKeyringError(
1 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/keyring/core.py:60 in set_password
58│ def set_password(service_name: str, username: str, password: str) -> None:
59│ """Set password for the user in the specified service."""
→ 60│ get_keyring().set_password(service_name, username, password)
61│
62│
PasswordSetError
Can't store password on keychain: (-25299, 'Unknown Error')
at ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/keyring/backends/macOS/__init__.py:44 in set_password
40│ api.set_generic_password(self.keychain, service, username, password)
41│ except api.KeychainDenied as e:
42│ raise KeyringLocked("Can't store password on keychain: " "{}".format(e))
43│ except api.Error as e:
→ 44│ raise PasswordSetError("Can't store password on keychain: " "{}".format(e))
45│
46│ def get_password(self, service, username):
47│ if username is None:
48│ username = ''
The following error occurred when trying to handle this error:
Stack trace:
9 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:329 in run
327│
328│ try:
→ 329│ exit_code = self._run(io)
330│ except Exception as e:
331│ if not self._catch_exceptions:
8 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/poetry/console/application.py:185 in _run
183│ self._load_plugins(io)
184│
→ 185│ exit_code: int = super()._run(io)
186│ return exit_code
187│
7 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:423 in _run
421│ io.input.set_stream(stream)
422│
→ 423│ exit_code = self._run_command(command, io)
424│ self._running_command = None
425│
6 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:465 in _run_command
463│
464│ if error is not None:
→ 465│ raise error
466│
467│ return event.exit_code
5 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:449 in _run_command
447│
448│ if event.command_should_run():
→ 449│ exit_code = command.run(io)
450│ else:
451│ exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED
4 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/cleo/commands/base_command.py:119 in run
117│ io.input.validate()
118│
→ 119│ status_code = self.execute(io)
120│
121│ if status_code is None:
3 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/cleo/commands/command.py:83 in execute
81│
82│ try:
→ 83│ return self.handle()
84│ except KeyboardInterrupt:
85│ return 1
2 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/poetry/console/commands/config.py:267 in handle
265│ password = values[1]
266│
→ 267│ password_manager.set_http_password(m.group(2), username, password)
268│ elif m.group(1) == "pypi-token":
269│ if len(values) != 1:
1 ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/password_manager.py:219 in set_http_password
217│ auth["password"] = password
218│ else:
→ 219│ self.keyring.set_password(name, username, password)
220│
221│ self._config.auth_config_source.add_property(f"http-basic.{name}", auth)
PoetryKeyringError
Unable to store the password for poetry-repository-prodigy in the key ring: Can't store password on keychain: (-25299, 'Unknown Error')
at ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/password_manager.py:87 in set_password
83│
84│ try:
85│ keyring.set_password(name, username, password)
86│ except (RuntimeError, keyring.errors.KeyringError) as e:
→ 87│ raise PoetryKeyringError(
88│ f"Unable to store the password for {name} in the key ring: {e}"
89│ )
90│
91│ def delete_password(self, name: str, username: str) -> None:
I know this issue is getting out of hand but I'm not sure if these errors are all linked to eachother?
There is also an ongoing thread here.
The keyring error is based on the fact the macOS keyring doesn't let you store an internet password as a username only. Despite the fact we were unable to set things in the keyring/threw an exception, we fell back to setting auth.toml successfully.
What is more interesting is that the solver crashes on an assertion -- either our typing is incorrect, or prodigy wheels are malformed.
I suspect @radoering and @dimbleby will be interested in looking at this. The Poetry team will need access to these wheels to introspect them. You can provide them privately by emailing me (you may find my email in Git history) and I can distribute them to the relevant parties -- alternatively, have the Prodigy people reach out via the same method.
Also, please provide a sanitized pyproject.toml per the issue template -- thanks!
Also, please provide a sanitized pyproject.toml per the issue template -- thanks!
@neersighted I thought I did already? (or was it just the sanitized bit - marking it as toml?)
I wasn't sure if your minimal pyproject.toml reproduced with only prodigy, or if there were more dependencies. Typically we want to see the actual pyproject.toml (or as close as we can get) when there are solver errors. However in this case it looks like just prodigy might be enough?
Yes the minimal example is enough - I can't get to 403 Client Error: Forbidden for url: https://s3.eu-... anymore though. That only happened once on my first attempt
It would have happened once due to not having credentials set up properly -- it should not reproduce now that they're in auth.toml.
I'm not so sure actually. Since the credentials is needed to get the wheel link - see #4497.
The error occurs due to poetry passing my HTTP Basic auth credentials to the AWS S3 pre-signed URL. AWS throws the error saying there can only be 1 auth method (which is the pre-signed URL) but Basic Auth credentials were also passed
So there might be multiple problems here...
I don't think that is at all what is going on -- that wouldn't result in a 403, and the authenticator code has been pretty much replaced. Poetry is clearly able to grab the wheel given it can count releases and chokes on solving for incompatibility.
The keyring error is based on the fact the macOS keyring doesn't let you store an internet password as a username only.
Btw. I get the same error if I try to add a password as well, i.e. Unable to store the password for poetry-repository-prodigy in the key ring: Can't store password on keychain: (-25299, 'Unknown Error')
I'm not sure why your keyring is broken, but you can isolate it by turning off support and forcing the auth.toml fallback: see #1917 for methods to do so.
Ah so I can go to Keychain Access in macOS and delete the keyring from there. Then I can add it again.
Now it becomes really strange though. Instead of doing poetry config http-basic.prodigy <my-licence-key> "" I did
poetry config http-basic.prodigy <my-licence-key>
and then just clicked enter twice to confirm an empty password. After this I was able to install prodigy in my main project!?
However I still can't install prodigy this way in a fresh project. Very strange
I suggest we focus on the solver issue here, since once we solve that it's easier to isolate and open issues for other problems. When reproducing, I'd always suggest using a container to isolate local configuration issues -- e.g. can you try to solve with a fresh container?
Sounds like a good idea!
A docker container? Do you have an image template you use or?
The python:3.x library images on the Docker Hub are great as they use Python.org builds with no patches, and include most of the software you want (e.g. git, a C compiler, etc) out of the box.
So I made a container with
FROM python:3.9.9
ENV POETRY_HOME=/usr/local
RUN curl -sSL https://install.python-poetry.org | python3 -
Then I ran a interactive session and did the following
mkdir project && cd project
poetry init
poetry source add prodigy "https://download.prodi.gy"
poetry config http-basic.prodigy <my-licence-key>
poetry add --source prodigy prodigy
And it just worked. So I'm not sure what's going on in my other case there...
When I added the auth I got this
> poetry config http-basic.prodigy <my-licence>
Password:
Using a plaintext file to store credentials
Invalid type <class 'NoneType'>
but it worked never the less
Hi @NixBiks , did you ever figure out why the Invalid type <class 'NoneType'> appeared? I'm running into some some issues with in builds because of it.
@mrcontre If you see the detailed error by adding -vvv to the command (e.g. poetry -vvv config ...), then it shows the error Invalid type <class 'NoneType'> itself comes from the tomlkit library (see related issue), but the mistake is actually much simpler - it happens when you try set the password config with an empty value.
I had the exact same issue in CI only, but it turns out it was because the environment variable wasn't set, e.g. when running poetry config http-basic.local_registry username $REGISTRY_PASSWORD and REGISTRY_PASSWORD wasn't set.
The error message could definitely be more helpful, although this seems to be a bit of an edge-case because if there's an interactive terminal then the command would try and allow the password to be entered via prompt (you can duplicate the issue then by just pressing enter twice at that prompt).
Thanks @samuller, I eventually found the issue and if I'm not mistaken it had something to do with an unset environment variable as you pointed out. A more specific error message would definitely help!