Poetry fails to create lock file when specifying 'linux' platform: Package('pywin32', '306')

[jeertmans]

• Poetry version: 1.7.0
• Python version: 3.10.12
• OS version and name: Ubuntu 22.04
• pyproject.toml: https://gist.github.com/jeertmans/8416565ee6dbbb7ebc067899315a0acb

• [x] I am on the latest stable Poetry version, installed using a recommended method.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] I have consulted the FAQ and blog for any relevant entries or release notes.
• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

Hello, I am developing a Python package, for which I have a dependency which is only available on Linux. This dependency is only used in GitHub workflows, hence the group name (see Gist).

When restricting the platform to "linux", I guess the following error message (full traceback below) when calling poetry lock or poetry update:

Package('pywin32', '306')

caused by changing:

- open3d-cpu = {version = "^0.17.0", python = "<3.11"}
+ open3d-cpu = {version = "^0.17.0", platform = "linux", python = "<3.11"}

If I do not specify the platform, then I cannot install my package on Windows or macOS (because open3d-cpu does not exist on those platforms).

I guess this is related to #5027, but the proposed solution (using markers) does not work properly.

For information, here is the PR that initiated the build of open3d-cpu linux-only wheels: https://github.com/isl-org/Open3D/pull/5902.

Conflicting package

After searching a bit, the error seems to be caused in pair with myst-nb, but I don't see why at the first point. For information, here is a link to the pyproject.toml of myst-nb.

Stack trace:

4 ~/.local/lib/python3.10/site-packages/poetry/puzzle/solver.py:155 in _solve 153│ 154│ try: → 155│ result = resolve_version(self._package, self._provider) 156│ 157│ packages = result.packages

3 ~/.local/lib/python3.10/site-packages/poetry/mixology/init.py:18 in resolve_version 16│ solver = VersionSolver(root, provider) 17│ → 18│ return solver.solve() 19│

2 ~/.local/lib/python3.10/site-packages/poetry/mixology/version_solver.py:164 in solve 162│ while next is not None: 163│ self._propagate(next) → 164│ next = self._choose_package_version() 165│ 166│ return self._result()

1 ~/.local/lib/python3.10/site-packages/poetry/mixology/version_solver.py:503 in _choose_package_version 501│ package = locked 502│ → 503│ package = self._provider.complete_package(package) 504│ 505│ conflict = False

OverrideNeeded

({Package('pandas', '2.1.3'): {'numpy': <Dependency numpy (>=1.22.4,<2)>}}, {Package('pandas', '2.1.3'): {'numpy': <Dependency numpy (>=1.23.2,<2)>}}, {Package('pandas', '2.1.3'): {'numpy': <Dependency numpy (>=1.26.0,<2)>}})

at ~/.local/lib/python3.10/site-packages/poetry/puzzle/provider.py:660 in complete_package 656│ current_overrides.update({dependency_package: package_overrides}) 657│ overrides.append(current_overrides) 658│ 659│ if overrides: → 660│ raise OverrideNeeded(*overrides) 661│ 662│ # Modifying dependencies as needed 663│ clean_dependencies = [] 664│ for dep in dependencies:

The following error occurred when trying to handle this error:

Stack trace:

13 ~/.local/lib/python3.10/site-packages/cleo/application.py:327 in run 325│ 326│ try: → 327│ exit_code = self._run(io) 328│ except BrokenPipeError: 329│ # If we are piped to another process, it may close early and send a

12 ~/.local/lib/python3.10/site-packages/poetry/console/application.py:190 in _run 188│ self._load_plugins(io) 189│ → 190│ exit_code: int = super()._run(io) 191│ return exit_code 192│

11 ~/.local/lib/python3.10/site-packages/cleo/application.py:431 in _run 429│ io.input.interactive(interactive) 430│ → 431│ exit_code = self._run_command(command, io) 432│ self._running_command = None 433│

10 ~/.local/lib/python3.10/site-packages/cleo/application.py:473 in _run_command 471│ 472│ if error is not None: → 473│ raise error 474│ 475│ return terminate_event.exit_code

9 ~/.local/lib/python3.10/site-packages/cleo/application.py:457 in _run_command 455│ 456│ if command_event.command_should_run(): → 457│ exit_code = command.run(io) 458│ else: 459│ exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

8 ~/.local/lib/python3.10/site-packages/cleo/commands/base_command.py:117 in run 115│ io.input.validate() 116│ → 117│ return self.execute(io) or 0 118│ 119│ def merge_application_definition(self, merge_args: bool = True) -> None:

7 ~/.local/lib/python3.10/site-packages/cleo/commands/command.py:61 in execute 59│ 60│ try: → 61│ return self.handle() 62│ except KeyboardInterrupt: 63│ return 1

6 ~/.local/lib/python3.10/site-packages/poetry/console/commands/lock.py:55 in handle 53│ self.installer.lock(update=not self.option("no-update")) 54│ → 55│ return self.installer.run() 56│

5 ~/.local/lib/python3.10/site-packages/poetry/installation/installer.py:104 in run 102│ self.verbose(True) 103│ → 104│ return self._do_install() 105│ 106│ def dry_run(self, dry_run: bool = True) -> Installer:

4 ~/.local/lib/python3.10/site-packages/poetry/installation/installer.py:241 in _do_install 239│ source_root=self._env.path.joinpath("src") 240│ ): → 241│ ops = solver.solve(use_latest=self._whitelist).calculate_operations() 242│ else: 243│ self._io.write_line("Installing dependencies from lock file")

3 ~/.local/lib/python3.10/site-packages/poetry/puzzle/solver.py:72 in solve 70│ with self._progress(), self._provider.use_latest_for(use_latest or []): 71│ start = time.time() → 72│ packages, depths = self._solve() 73│ end = time.time() 74│

2 ~/.local/lib/python3.10/site-packages/poetry/puzzle/solver.py:159 in _solve 157│ packages = result.packages 158│ except OverrideNeeded as e: → 159│ return self._solve_in_compatibility_mode(e.overrides) 160│ except SolveFailure as e: 161│ raise SolverProblemError(e)

1 ~/.local/lib/python3.10/site-packages/poetry/puzzle/solver.py:133 in _solve_in_compatibility_mode 131│ ) 132│ self._provider.set_overrides(override) → 133│ _packages, _depths = self._solve() 134│ for index, package in enumerate(_packages): 135│ if package not in packages:

KeyError

Package('pywin32', '306')

at ~/.local/lib/python3.10/site-packages/poetry/puzzle/solver.py:194 in _solve 190│ # because it includes relevant extras 191│ _dep.marker = dep.marker 192│ else: 193│ final_packages.append(package) → 194│ depths.append(results[package]) 195│ 196│ # Return the packages in their original order with associated depths 197│ return final_packages, depths 198│

[radoering]

This might be some variant of #5506 although the resulting error is quite different. At least I am able to lock the example with #6969. However, that probably does not help much because #6969 makes the solver very slow. (It took about 6 minutes to lock this example.)

[george-zubrienko]

I'd say if we can avoid resolving Windows-related dependencies when we do not need them, would be great. We had this issue when running update on linux/macos envs, fix is to add an upper boundary that removes the faulty 306 release of whatever that pywin32 is...

[tool.poetry.dev-dependencies]
pywin32 = { version = "<306", platform = "win32" }