python-monorepo icon indicating copy to clipboard operation
python-monorepo copied to clipboard

Transitive editable deps fail

Open haf opened this issue 3 years ago • 1 comments

If you have svc/A, libs/a, libs/b and A depends on both a and b, and b depends on a, installing b via the dockerfiles in this repo (the wheel output from building it), with:

 > [4/6] RUN pip install *.whl && rm -rf *.whl:
#6 0.670 Processing /a-0.4.0-py3-none-any.whl
#6 0.681 Processing /c-0.4.0-py3-none-any.whl
#6 0.692 Processing /b-0.4.0-py3-none-any.whl
#6 0.765 Collecting psycopg[binary]<4.0,>=3.0
#6 0.795   Downloading psycopg-3.0.7-py3-none-any.whl (141 kB)
#snip
#6 9.185 Collecting numpy<2.0.0,>=1.21.1
#6 9.198   Downloading numpy-1.21.5-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.7 MB)
#6 10.61 ERROR: Exception:
#6 10.61 Traceback (most recent call last):
#6 10.61   File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3021, in _dep_map
#6 10.61     return self.__dep_map
#6 10.61   File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2815, in __getattr__
#6 10.61     raise AttributeError(attr)
#6 10.61 AttributeError: _DistInfoDistribution__dep_map
#6 10.61
#6 10.61 During handling of the above exception, another exception occurred:
#6 10.61
#6 10.61 Traceback (most recent call last):
#6 10.61   File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3101, in __init__
#6 10.61     super(Requirement, self).__init__(requirement_string)
#6 10.61   File "/usr/local/lib/python3.9/site-packages/pip/_vendor/packaging/requirements.py", line 117, in __init__
#6 10.61     raise InvalidRequirement(f"Invalid URL: {req.url}")
#6 10.61 pip._vendor.packaging.requirements.InvalidRequirement: Invalid URL: ../../libs/libpg
#6 10.61
#6 10.61 During handling of the above exception, another exception occurred:
#6 10.61
#6 10.61 Traceback (most recent call last):
#6 10.61   File "/usr/local/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 164, in exc_logging_wrapper
#6 10.61     status = run_func(*args)
#6 10.61   File "/usr/local/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
#6 10.61     return func(self, options, args)
#6 10.61   File "/usr/local/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 338, in run
#6 10.61     requirement_set = resolver.resolve(
#6 10.61   File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
#6 10.61     result = self._result = resolver.resolve(
#6 10.61   File "/usr/local/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 482, in resolve
#6 10.61     state = resolution.resolve(requirements, max_rounds=max_rounds)
#6 10.61   File "/usr/local/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 374, in resolve
#6 10.61     failure_causes = self._attempt_to_pin_criterion(name)
#6 10.61   File "/usr/local/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 214, in _attempt_to_pin_criterion
#6 10.61     criteria = self._get_updated_criteria(candidate)
#6 10.61   File "/usr/local/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 204, in _get_updated_criteria
#6 10.61     for requirement in self._p.get_dependencies(candidate=candidate):
#6 10.61   File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/provider.py", line 204, in get_dependencies
#6 10.61     return [r for r in candidate.iter_dependencies(with_requires) if r is not None]
#6 10.61   File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/provider.py", line 204, in <listcomp>
#6 10.61     return [r for r in candidate.iter_dependencies(with_requires) if r is not None]
#6 10.61   File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 236, in iter_dependencies
#6 10.61     requires = self.dist.iter_dependencies() if with_requires else ()
#6 10.61   File "/usr/local/lib/python3.9/site-packages/pip/_internal/metadata/pkg_resources.py", line 91, in iter_dependencies
#6 10.61     return self._dist.requires(extras)
#6 10.61   File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2736, in requires
#6 10.61     dm = self._dep_map
#6 10.61   File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3023, in _dep_map
#6 10.61     self.__dep_map = self._compute_dependencies()
#6 10.61   File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3033, in _compute_dependencies
#6 10.61     reqs.extend(parse_requirements(req))
#6 10.61   File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3094, in parse_requirements
#6 10.61     yield Requirement(line)
#6 10.61   File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3103, in __init__
#6 10.61     raise RequirementParseError(str(e))
#6 10.61 pip._vendor.pkg_resources.RequirementParseError: Invalid URL: ../../libs/a
------
executor failed running [/bin/sh -c pip install *.whl && rm -rf *.whl]: exit code: 2
make: *** [container] Error 1

haf avatar Dec 22 '21 23:12 haf

I think order of installation of dependencies matters. I.e in your example a should be installed first, before b. This means the dependency tree matters and we need to install in topological order. Seems like re-creating make :( Wish poetry would just do it automatically.

KholdStare avatar Mar 07 '22 22:03 KholdStare