python-poetry
poetry-core 2.0.0 can no longer build when the sourcetree contains symlinks outside itself
Description
Since the release of poetry-core v2.0.0, twine can no longer build a poetry package whose source tree contains symlinks outside itself. Previously it could. Now it fails with errors like
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
main()
File "/usr/local/lib/python3.10/dist-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in main
json_out["return_val"] = hook(**hook_input["kwargs"])
File "/usr/local/lib/python3.10/dist-packages/pyproject_hooks/_in_process/_in_process.py", line 280, in build_wheel
return _build_backend().build_wheel(
File "/tmp/build-env-cf76kflm/lib/python3.10/site-packages/poetry/core/masonry/api.py", line 58, in build_wheel
return WheelBuilder.make_in(
File "/tmp/build-env-cf76kflm/lib/python3.10/site-packages/poetry/core/masonry/builders/wheel.py", line 91, in make_in
wb.build(target_dir=directory)
File "/tmp/build-env-cf76kflm/lib/python3.10/site-packages/poetry/core/masonry/builders/wheel.py", line 131, in build
self._copy_module(zip_file)
File "/tmp/build-env-cf76kflm/lib/python3.10/site-packages/poetry/core/masonry/builders/wheel.py", line 308, in _copy_module
self._add_file(wheel, file.path, file.relative_to_target_root())
File "/tmp/build-env-cf76kflm/lib/python3.10/site-packages/poetry/core/masonry/builders/builder.py", line 374, in relative_to_target_root
path = self.relative_to_source_root()
File "/tmp/build-env-cf76kflm/lib/python3.10/site-packages/poetry/core/masonry/builders/builder.py", line 371, in relative_to_source_root
return self.path.relative_to(self.source_root)
File "/usr/lib/python3.10/pathlib.py", line 818, in relative_to
raise ValueError("{!r} is not in the subpath of {!r}"
ValueError: '/true/location/of/myschema.json' is not in the subpath of '/mypackage/src' OR one path is relative and the other is absolute.
Is this deliberate? I realise it is an edge case, and probably there are arguments for disallowing these symlinks.
(It doesn't seem to be possible for me to raise an issue against poetry-core itself.)
Workarounds
Poetry Installation Method
other
Operating System
Ubuntu 22.04
Poetry Version
poetry-core v2.0.0
Poetry Configuration
-
Python Sysconfig
No response
Example pyproject.toml
No response
Poetry Runtime Logs
-
Are you certain that this has changed? there are several existing issues reporting the same behaviour
either way it looks intended and correct
Well, we've had these symlinks in place for about 6 months, and an unchanged CI pipeline building our package. The failures coincide with the release of the new version.
If it is confirmed that this is intended and correct, we'll just have to find a way to get rid of these symlinks. As I said, I can certainly see why you wouldn't want to support this kind of thing.
nevertheless I tried for myself with older versions of poetry / poetry-core - and indeed the same error is raised.
So either there is something more subtle about what you are doing which we have not identified - or you have misunderstood what has changed
It does looks like tools like setuptools allow for symlinks in the source tree. Whether this is good or bad, I am not sure. To me, it feels like you could potentially end up with completely different files when you attempt to rebuild a package and or different failures.
Maybe it is worth considering the adding the ability to allow for includes to explicitly allow symlinks if a project really wants to do it.
Supporting symlinks feels like quite the can of worms: and I continue to claim that this hasn't actually changed recently at all. (Though I don't know how to explain OP's report that something has changed!)
If I am right and this has never been supported then I suppose this could become a feature request. But it sounds as though even the reporter doesn't think this is a particularly great idea, my gut would be to turn that down.
If I am wrong and something used to work but now doesn't then it probably depends what that investigation shows: in that case maybe something should be reverted, or maybe it's better now anyway.