poetry
poetry copied to clipboard
python-poetry
Poetry fails to resolve dependencies that pull from a source
Description
I have a pyproject.toml that pulls another remote package that has a source for torch-scatter
the torch scatter implementation is as such
[tool.poetry.dependencies]
python = "^3.10"
torch-scatter = { version = "=2.1.2+pt22cpu", source = "scatter-cpu" }
[[tool.poetry.source]]
name = "scatter-cpu"
url = "https://data.pyg.org/whl/torch-2.2.0%2Bcpu.html"
if i build that project locally it installs fine. but if I call to build from another pyproject it gives error Repository "scatter-cpu" does not exist. when attempting to do a poetry lock
Workarounds
right now i have to add the source from the dependency:
[[tool.poetry.source]]
name = "scatter-cpu"
url = "https://data.pyg.org/whl/torch-2.2.0%2Bcpu.html"
to my top level pyproject.toml or it wont build.
Poetry Installation Method
pipx
Operating System
Fedora 38
Poetry Version
1.8.2
Poetry Configuration
cache-dir = "/home/dev/.cache/pypoetry"
experimental.system-git-client = false
installer.max-workers = null
installer.modern-installation = true
installer.no-binary = null
installer.parallel = true
keyring.enabled = true
repositories.scatter-cpu.url = "https://data.pyg.org/whl/torch-2.2.0%2Bcpu.html"
repositories.torch-cpu.url = "https://download.pytorch.org/whl/cpu"
repositories.torch-cu118.url = "https://download.pytorch.org/whl/cu118"
solver.lazy-wheel = true
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.no-setuptools = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}/virtualenvs" # /home/dev/.cache/pypoetry/virtualenvs
virtualenvs.prefer-active-python = false
virtualenvs.prompt = "{project_name}-py{python_version}"
warnings.export = true
Python Sysconfig
No response
Example pyproject.toml
[tool.poetry.dependencies]
python = "^3.10"
torch-scatter = { version = "=2.1.2+pt22cpu", source = "scatter-cpu" }
[[tool.poetry.source]]
name = "scatter-cpu"
url = "https://data.pyg.org/whl/torch-2.2.0%2Bcpu.html"
Poetry Runtime Logs
Stack trace:
4 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/puzzle/solver.py:154 in _solve
152│
153│ try:
→ 154│ result = resolve_version(self._package, self._provider)
155│
156│ packages = result.packages
3 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/mixology/__init__.py:18 in resolve_version
16│ solver = VersionSolver(root, provider)
17│
→ 18│ return solver.solve()
19│
2 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/mixology/version_solver.py:175 in solve
173│ while next is not None:
174│ self._propagate(next)
→ 175│ next = self._choose_package_version()
176│
177│ return self._result()
1 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/mixology/version_solver.py:514 in _choose_package_version
512│ package = locked
513│
→ 514│ package = self._provider.complete_package(package)
515│
516│ conflict = False
OverrideNeeded
({Package('gt4sd-inference-all', '0.1.0', source_type='git', source_url='xxxx, source_reference='main', source_resolved_reference='', source_subdirectory='openad-model-inference/gt4sd_inference_all'): {'torch': <Dependency torch (>=1.7.1)>}}, {Package('gt4sd-inference-all', '0.1.0', source_type='git', source_url='xxxx', source_reference='main', source_resolved_reference='', source_subdirectory='openad-model-inference/gt4sd_inference_all'): {'torch': <Dependency torch (<empty>)>}}, {Package('gt4sd-inference-all', '0.1.0', source_type='git', source_url='xxxx', source_reference='main', source_resolved_reference='', source_subdirectory='openad-model-inference/gt4sd_inference_all'): {'torch': <Dependency torch (>=1.7.1+cu118)>}}, {Package('gt4sd-inference-all', '0.1.0', source_type='git', source_url='xxxx', source_reference='main', source_resolved_reference='', source_subdirectory='openad-model-inference/gt4sd_inference_all'): {'torch': <Dependency torch (>=1.7.1+cpu)>}})
at ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/puzzle/provider.py:653 in complete_package
649│ current_overrides.update({package: package_overrides})
650│ overrides.append(current_overrides)
651│
652│ if overrides:
→ 653│ raise OverrideNeeded(*overrides)
654│
655│ # Modifying dependencies as needed
656│ clean_dependencies = []
657│ for dep in dependencies:
The following error occurred when trying to handle this error:
Stack trace:
4 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/puzzle/solver.py:154 in _solve
152│
153│ try:
→ 154│ result = resolve_version(self._package, self._provider)
155│
156│ packages = result.packages
3 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/mixology/__init__.py:18 in resolve_version
16│ solver = VersionSolver(root, provider)
17│
→ 18│ return solver.solve()
19│
2 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/mixology/version_solver.py:175 in solve
173│ while next is not None:
174│ self._propagate(next)
→ 175│ next = self._choose_package_version()
176│
177│ return self._result()
1 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/mixology/version_solver.py:514 in _choose_package_version
512│ package = locked
513│
→ 514│ package = self._provider.complete_package(package)
515│
516│ conflict = False
OverrideNeeded
{Package('gt4sd-inference-all', '0.1.0', source_type='git', source_url='xxxx, source_reference='main', source_resolved_reference='', source_subdirectory='openad-model-inference/gt4sd_inference_all'): {'torch': <Dependency torch (<empty>)>, 'torch-scatter': <URLDependency torch-scatter (<empty>) @ https://data.pyg.org/whl/torch-2.2.0%2Bcpu/torch_scatter-2.1.2%2Bpt22cpu-cp310-cp310-linux_x86_64.whl>}}
at ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/puzzle/provider.py:653 in complete_package
649│ current_overrides.update({package: package_overrides})
650│ overrides.append(current_overrides)
651│
652│ if overrides:
→ 653│ raise OverrideNeeded(*overrides)
654│
655│ # Modifying dependencies as needed
656│ clean_dependencies = []
657│ for dep in dependencies:
The following error occurred when trying to handle this error:
Stack trace:
4 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/puzzle/solver.py:154 in _solve
152│
153│ try:
→ 154│ result = resolve_version(self._package, self._provider)
155│
156│ packages = result.packages
3 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/mixology/__init__.py:18 in resolve_version
16│ solver = VersionSolver(root, provider)
17│
→ 18│ return solver.solve()
19│
2 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/mixology/version_solver.py:175 in solve
173│ while next is not None:
174│ self._propagate(next)
→ 175│ next = self._choose_package_version()
176│
177│ return self._result()
1 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/mixology/version_solver.py:514 in _choose_package_version
512│ package = locked
513│
→ 514│ package = self._provider.complete_package(package)
515│
516│ conflict = False
OverrideNeeded
{Package('gt4sd-inference-all', '0.1.0', source_type='git', source_url='xxxx, source_reference='main', source_resolved_reference=', source_subdirectory='openad-model-inference/gt4sd_inference_all'): {'torch': <Dependency torch (<empty>)>, 'torch-scatter': <URLDependency torch-scatter (<empty>) @ https://data.pyg.org/whl/torch-2.2.0%2Bcpu/torch_scatter-2.1.2%2Bpt22cpu-cp310-cp310-linux_x86_64.whl>, 'torch-cluster': <URLDependency torch-cluster (<empty>) @ https://data.pyg.org/whl/torch-2.2.0%2Bcpu/torch_cluster-1.6.3-cp310-cp310-macosx_11_0_x86_64.whl>}}
at ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/puzzle/provider.py:653 in complete_package
649│ current_overrides.update({package: package_overrides})
650│ overrides.append(current_overrides)
651│
652│ if overrides:
→ 653│ raise OverrideNeeded(*overrides)
654│
655│ # Modifying dependencies as needed
656│ clean_dependencies = []
657│ for dep in dependencies:
The following error occurred when trying to handle this error:
Stack trace:
4 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/puzzle/solver.py:154 in _solve
152│
153│ try:
→ 154│ result = resolve_version(self._package, self._provider)
155│
156│ packages = result.packages
3 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/mixology/__init__.py:18 in resolve_version
16│ solver = VersionSolver(root, provider)
17│
→ 18│ return solver.solve()
19│
2 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/mixology/version_solver.py:175 in solve
173│ while next is not None:
174│ self._propagate(next)
→ 175│ next = self._choose_package_version()
176│
177│ return self._result()
1 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/mixology/version_solver.py:514 in _choose_package_version
512│ package = locked
513│
→ 514│ package = self._provider.complete_package(package)
515│
516│ conflict = False
OverrideNeeded
{Package('gt4sd-inference-all', '0.1.0', source_type='git', source_url='xxxx', source_reference='main', source_resolved_reference='', source_subdirectory='openad-model-inference/gt4sd_inference_all'): {'torch': <Dependency torch (<empty>)>, 'torch-scatter': <URLDependency torch-scatter (<empty>) @ https://data.pyg.org/whl/torch-2.2.0%2Bcpu/torch_scatter-2.1.2%2Bpt22cpu-cp310-cp310-linux_x86_64.whl>, 'torch-cluster': <URLDependency torch-cluster (<empty>) @ https://data.pyg.org/whl/torch-2.2.0%2Bcpu/torch_cluster-1.6.3-cp310-cp310-macosx_11_0_x86_64.whl>}, Package('gt4sd-inference-torch-drug', '0.1.0', source_type='directory', source_url='/home/dev/.cache/pypoetry/virtualenvs/openad-model-generation-service-W9SyLlNI-py3.10/src/Open-AD-Model-Service/openad-model-inference/gt4sd_torch'): {'torch': <Dependency torch (<empty>)>}}
at ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/puzzle/provider.py:653 in complete_package
649│ current_overrides.update({package: package_overrides})
650│ overrides.append(current_overrides)
651│
652│ if overrides:
→ 653│ raise OverrideNeeded(*overrides)
654│
655│ # Modifying dependencies as needed
656│ clean_dependencies = []
657│ for dep in dependencies:
The following error occurred when trying to handle this error:
Stack trace:
4 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/puzzle/solver.py:154 in _solve
152│
153│ try:
→ 154│ result = resolve_version(self._package, self._provider)
155│
156│ packages = result.packages
3 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/mixology/__init__.py:18 in resolve_version
16│ solver = VersionSolver(root, provider)
17│
→ 18│ return solver.solve()
19│
2 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/mixology/version_solver.py:175 in solve
173│ while next is not None:
174│ self._propagate(next)
→ 175│ next = self._choose_package_version()
176│
177│ return self._result()
1 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/mixology/version_solver.py:514 in _choose_package_version
512│ package = locked
513│
→ 514│ package = self._provider.complete_package(package)
515│
516│ conflict = False
OverrideNeeded
{Package('gt4sd-inference-all', '0.1.0', source_type='git', source_url='xxxx', source_reference='main', source_resolved_reference='', source_subdirectory='openad-model-inference/gt4sd_inference_all'): {'torch': <Dependency torch (<empty>)>, 'torch-scatter': <URLDependency torch-scatter (<empty>) @ https://data.pyg.org/whl/torch-2.2.0%2Bcpu/torch_scatter-2.1.2%2Bpt22cpu-cp310-cp310-linux_x86_64.whl>, 'torch-cluster': <URLDependency torch-cluster (<empty>) @ https://data.pyg.org/whl/torch-2.2.0%2Bcpu/torch_cluster-1.6.3-cp310-cp310-macosx_11_0_x86_64.whl>}, Package('gt4sd-inference-torch-drug', '0.1.0', source_type='directory', source_url='/home/dev/.cache/pypoetry/virtualenvs/openad-model-generation-service-W9SyLlNI-py3.10/src/Open-AD-Model-Service/openad-model-inference/gt4sd_torch'): {'torch': <Dependency torch (<empty>)>, 'torch-cluster': <URLDependency torch-cluster (<empty>) @ https://data.pyg.org/whl/torch-2.2.0%2Bcpu/torch_cluster-1.6.3-cp310-cp310-macosx_11_0_x86_64.whl>}}
at ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/puzzle/provider.py:653 in complete_package
649│ current_overrides.update({package: package_overrides})
650│ overrides.append(current_overrides)
651│
652│ if overrides:
→ 653│ raise OverrideNeeded(*overrides)
654│
655│ # Modifying dependencies as needed
656│ clean_dependencies = []
657│ for dep in dependencies:
The following error occurred when trying to handle this error:
Stack trace:
31 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/cleo/application.py:327 in run
325│
326│ try:
→ 327│ exit_code = self._run(io)
328│ except BrokenPipeError:
329│ # If we are piped to another process, it may close early and send a
30 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/console/application.py:190 in _run
188│ self._load_plugins(io)
189│
→ 190│ exit_code: int = super()._run(io)
191│ return exit_code
192│
29 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/cleo/application.py:431 in _run
429│ io.input.interactive(interactive)
430│
→ 431│ exit_code = self._run_command(command, io)
432│ self._running_command = None
433│
28 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/cleo/application.py:473 in _run_command
471│
472│ if error is not None:
→ 473│ raise error
474│
475│ return terminate_event.exit_code
27 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/cleo/application.py:457 in _run_command
455│
456│ if command_event.command_should_run():
→ 457│ exit_code = command.run(io)
458│ else:
459│ exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED
26 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/cleo/commands/base_command.py:117 in run
115│ io.input.validate()
116│
→ 117│ return self.execute(io) or 0
118│
119│ def merge_application_definition(self, merge_args: bool = True) -> None:
25 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/cleo/commands/command.py:61 in execute
59│
60│ try:
→ 61│ return self.handle()
62│ except KeyboardInterrupt:
63│ return 1
24 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/console/commands/lock.py:55 in handle
53│ self.installer.lock(update=not self.option("no-update"))
54│
→ 55│ return self.installer.run()
56│
23 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/installation/installer.py:104 in run
102│ self.verbose(True)
103│
→ 104│ return self._do_install()
105│
106│ def dry_run(self, dry_run: bool = True) -> Installer:
22 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/installation/installer.py:241 in _do_install
239│ source_root=self._env.path.joinpath("src")
240│ ):
→ 241│ ops = solver.solve(use_latest=self._whitelist).calculate_operations()
242│ else:
243│ self._io.write_line("Installing dependencies from lock file")
21 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/puzzle/solver.py:71 in solve
69│ with self._progress(), self._provider.use_latest_for(use_latest or []):
70│ start = time.time()
→ 71│ packages, depths = self._solve()
72│ end = time.time()
73│
... Previous 2 frames repeated 3 times
14 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/puzzle/solver.py:158 in _solve
156│ packages = result.packages
157│ except OverrideNeeded as e:
→ 158│ return self._solve_in_compatibility_mode(e.overrides)
159│ except SolveFailure as e:
160│ raise SolverProblemError(e)
13 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/puzzle/solver.py:132 in _solve_in_compatibility_mode
130│ )
131│ self._provider.set_overrides(override)
→ 132│ _packages, _depths = self._solve()
133│ for index, package in enumerate(_packages):
134│ if package not in packages:
12 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/puzzle/solver.py:158 in _solve
156│ packages = result.packages
157│ except OverrideNeeded as e:
→ 158│ return self._solve_in_compatibility_mode(e.overrides)
159│ except SolveFailure as e:
160│ raise SolverProblemError(e)
11 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/puzzle/solver.py:132 in _solve_in_compatibility_mode
130│ )
131│ self._provider.set_overrides(override)
→ 132│ _packages, _depths = self._solve()
133│ for index, package in enumerate(_packages):
134│ if package not in packages:
10 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/puzzle/solver.py:154 in _solve
152│
153│ try:
→ 154│ result = resolve_version(self._package, self._provider)
155│
156│ packages = result.packages
9 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/mixology/__init__.py:18 in resolve_version
16│ solver = VersionSolver(root, provider)
17│
→ 18│ return solver.solve()
19│
8 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/mixology/version_solver.py:175 in solve
173│ while next is not None:
174│ self._propagate(next)
→ 175│ next = self._choose_package_version()
176│
177│ return self._result()
7 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/mixology/version_solver.py:493 in _choose_package_version
491│ return is_specific_marker, preference, -num_packages
492│
→ 493│ dependency = min(unsatisfied, key=_get_min)
494│
495│ locked = self._provider.get_locked(dependency)
6 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/mixology/version_solver.py:480 in _get_min
478│
479│ num_packages = len(
→ 480│ self._dependency_cache.search_for(
481│ dependency, self._solution.decision_level
482│ )
5 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/mixology/version_solver.py:112 in search_for
110│ # We could always use dependency.without_features() here,
111│ # but for performance reasons we only do it if necessary.
→ 112│ packages = self._search_for_cached(
113│ dependency.without_features() if dependency.features else dependency, key
114│ )
4 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/mixology/version_solver.py:93 in _search_for
91│ # additional results this time.
92│ if not packages:
→ 93│ packages = self._provider.search_for(dependency)
94│
95│ return packages
3 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/puzzle/provider.py:300 in search_for
298│ return PackageCollection(dependency, packages)
299│
→ 300│ packages = self._pool.find_packages(dependency)
301│
302│ packages.sort(
2 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/repositories/repository_pool.py:212 in find_packages
210│ repository_name = dependency.source_name
211│ if repository_name:
→ 212│ return self.repository(repository_name).find_packages(dependency)
213│
214│ packages: list[Package] = []
1 ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/repositories/repository_pool.py:136 in repository
134│
135│ def repository(self, name: str) -> Repository:
→ 136│ return self._get_prioritized_repository(name).repository
137│
138│ def get_priority(self, name: str) -> Priority:
IndexError
Repository "scatter-cpu" does not exist.
at ~/.local/share/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/repositories/repository_pool.py:145 in _get_prioritized_repository
141│ def _get_prioritized_repository(self, name: str) -> PrioritizedRepository:
142│ name = name.lower()
143│ if self.has_repository(name):
144│ return self._repositories[name]
→ 145│ raise IndexError(f'Repository "{name}" does not exist.')
146│
147│ def add_repository(
148│ self,
149│ repository: Repository,
this is expected, source information cannot be transferred via the standards-defined metadata.
please close
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}