python-poetry
`requires-plugins` breaks poetry installation on Windows
Description
I have a poetry project, which I install and test on Windows machine in GitHub Actions.
Recently, I tried using the new requires-plugins feature for poethepoet and poetry-plugin-export plugins installation.
The pyproject.toml file looks somewhat like this:
...
[tool.poetry.requires-plugins]
poethepoet = "^0.32.1"
poetry-plugin-export = "^1.9.0"
...
However, instead of expected installation, what I receive is an error:
Ensuring that the Poetry plugins required by the project are available...
The following Poetry plugins are required by the project but are not installed in Poetry's environment:
- poethepoet (>=0.32.1,<0.33.0)
- poetry-plugin-export (>=1.9.0,<2.0.0)
Installing Poetry plugins only for the current project...
Updating dependencies
Resolving dependencies...
Package operations: 4 installs, 0 updates, 0 removals
- Installing pastel (0.2.1)
- Installing pyyaml (6.0.2)
- Installing poethepoet (0.32.1)
- Installing poetry-plugin-export (1.9.0)
ValueError
path is on mount 'C:', start on mount 'D:'
at C:\hostedtoolcache\windows\Python\3.10.11\x64\lib\ntpath.py:747 in relpath
743| path_abs = abspath(normpath(path))
744| start_drive, start_rest = splitdrive(start_abs)
745| path_drive, path_rest = splitdrive(path_abs)
746| if normcase(start_drive) != normcase(path_drive):
> 747| raise ValueError("path is on mount %r, start on mount %r" % (
748| path_drive, start_drive))
749|
750| start_list = [x for x in start_rest.split(sep) if x]
751| path_list = [x for x in path_rest.split(sep) if x]
Cannot install poethepoet.
Failed to install required Poetry plugins
Workarounds
The plugins still can be installed traditionally, using poetry self add ....
However, I still would like to try the new feature... it works on Linux so nice!
Poetry Installation Method
pip
Operating System
windiws-latest (that is Windows Server 2022 OS Version: 10.0.20348 Build 2966)
Poetry Version
Poetry (version 2.0.1)
Poetry Configuration
cache-dir = "C:\\Users\\runneradmin\\AppData\\Local\\pypoetry\\Cache"
installer.max-workers = null
installer.no-binary = null
installer.only-binary = null
installer.parallel = true
installer.re-resolve = true
keyring.enabled = true
requests.max-retries = 0
solver.lazy-wheel = true
system-git-client = false
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}\\virtualenvs" # C:\Users\runneradmin\AppData\Local\pypoetry\Cache\virtualenvs
virtualenvs.prompt = "{project_name}-py{python_version}"
virtualenvs.use-poetry-python = false
Python Sysconfig
sysconfig.log
Platform: "win-amd64" Python version: "3.10" Current installation scheme: "nt" Paths: data = "C:\hostedtoolcache\windows\Python\3.10.11\x64" include = "C:\hostedtoolcache\windows\Python\3.10.11\x64\Include" platinclude = "C:\hostedtoolcache\windows\Python\3.10.11\x64\Include" platlib = "C:\hostedtoolcache\windows\Python\3.10.11\x64\Lib\site-packages" platstdlib = "C:\hostedtoolcache\windows\Python\3.10.11\x64\Lib" purelib = "C:\hostedtoolcache\windows\Python\3.10.11\x64\Lib\site-packages" scripts = "C:\hostedtoolcache\windows\Python\3.10.11\x64\Scripts" stdlib = "C:\hostedtoolcache\windows\Python\3.10.11\x64\Lib" Variables: BINDIR = "C:\hostedtoolcache\windows\Python\3.10.11\x64" BINLIBDEST = "C:\hostedtoolcache\windows\Python\3.10.11\x64\Lib" EXE = ".exe" EXT_SUFFIX = ".cp310-win_amd64.pyd" INCLUDEPY = "C:\hostedtoolcache\windows\Python\3.10.11\x64\Include" LIBDEST = "C:\hostedtoolcache\windows\Python\3.10.11\x64\Lib" SO = ".cp310-win_amd64.pyd" TZPATH = "" VERSION = "310" abiflags = "" base = "C:\hostedtoolcache\windows\Python\3.10.11\x64" exec_prefix = "C:\hostedtoolcache\windows\Python\3.10.11\x64" installed_base = "C:\hostedtoolcache\windows\Python\3.10.11\x64" installed_platbase = "C:\hostedtoolcache\windows\Python\3.10.11\x64" platbase = "C:\hostedtoolcache\windows\Python\3.10.11\x64" platlibdir = "lib" prefix = "C:\hostedtoolcache\windows\Python\3.10.11\x64" projectbase = "C:\hostedtoolcache\windows\Python\3.10.11\x64" py_version = "3.10.11" py_version_nodot = "310" py_version_nodot_plat = "310" py_version_short = "3.10" srcdir = "C:\hostedtoolcache\windows\Python\3.10.11\x64" userbase = "C:\Users\runneradmin\AppData\Roaming\Python"
Paste the output of 'python -m sysconfig', over this line.
Example pyproject.toml
[tool.poetry.requires-plugins]
poethepoet = "^0.32.1"
poetry-plugin-export = "^1.9.0"
Poetry Runtime Logs
This is likely a Windows file path issue like when the drive is a link and relpath is called.
Can you provide full logs with -vvv please?
I face the same problems on GitHub Actions workflow (where I don't fully control the environment).
I tried to add poetry-dynamic-versioning plugin:
[tool.poetry.requires-plugins]
poetry-dynamic-versioning = { version = ">=1.0.0,<2.0.0", extras = ["plugin"] }
My error is basically the same:
Run poetry sync --with test
Creating virtualenv skfp in D:\a\scikit-fingerprints\scikit-fingerprints\.venv
Ensuring that the Poetry plugins required by the project are available...
The following Poetry plugins are required by the project but are not installed in Poetry's environment:
- poetry-dynamic-versioning[plugin] (>=1.0.0,<2.0.0)
Installing Poetry plugins only for the current project...
Updating dependencies
Resolving dependencies...
Package operations: 4 installs, 0 updates, 0 removals
- Installing markupsafe (3.0.2)
- Installing dunamai (1.23.0)
- Installing jinja2 (3.1.5)
ValueError
path is on mount 'C:', start on mount 'D:'
at <frozen ntpath>:766 in relpath
Cannot install dunamai.
Failed to install required Poetry plugins
Error: Process completed with exit code 1.
Workflow definition: https://github.com/scikit-fingerprints/scikit-fingerprints/blob/eaa1499a12db49e128808f920df61d8004ef396e/.github/workflows/python-publish.yml.
Full stacktrace with -vvv option: https://github.com/scikit-fingerprints/scikit-fingerprints/actions/runs/12743574960/job/35513793864?pr=346
Thank you @j-adamczyk this is helpful. I can confirm the issue is as I suspected.
https://github.com/scikit-fingerprints/scikit-fingerprints/actions/runs/12743574960/job/35513793864?pr=346#step:7:1250
I am not entirely sure if this is something Poetry can fix as the failure is due to the way action runner sets up the mounts (for the user data directory). Might be able to make the code resilient to that, lets see.
Could you use abspath as a fallback when relpath fails? I found a similar case here: https://github.com/okken/pytest-check/pull/87/files
Probably, will have to check the impacted code first, I also would like to know if there are other places where we might be exposed to this issue.
Here's a full stack trace from a recent GA run on windows:
9 ~\poetry\venv\lib\site-packages\poetry\installation\executor.py:274 in _execute_operation
272|
273| try:
> 274| result = self._do_execute_operation(operation)
275| except EnvCommandError as e:
276| if e.e.returncode == -2:
8 ~\poetry\venv\lib\site-packages\poetry\installation\executor.py:385 in _do_execute_operation
383| return 0
384|
> 385| result: int = getattr(self, f"_execute_{method}")(operation)
386|
387| if result != 0:
7 ~\poetry\venv\lib\site-packages\poetry\installation\executor.py:509 in _execute_install
507|
508| def _execute_install(self, operation: Install | Update) -> int:
> 509| status_code = self._install(operation)
510|
511| self._save_url_reference(operation)
6 ~\poetry\venv\lib\site-packages\poetry\installation\executor.py:562 in _install
560| self._remove(operation.initial_package)
561|
> 562| self._wheel_installer.install(archive)
563| finally:
564| if cleanup_archive:
5 ~\poetry\venv\lib\site-packages\poetry\installation\wheel_installer.py:109 in install
107| )
108|
> 109| install(
110| source=source,
111| destination=destination,
4 ~\poetry\venv\lib\site-packages\installer\_core.py:131 in install
129|
130| written_records.append((root_scheme, RecordEntry(record_file_path, None, None)))
> 131| destination.finalize_installation(
132| scheme=root_scheme,
133| record_file_path=record_file_path,
3 ~\poetry\venv\lib\site-packages\installer\destinations.py:278 in finalize_installation
276|
277| record_list = list(records)
> 278| with construct_record_file(record_list, prefix_for_scheme) as record_stream:
279| self.write_to_fs(
280| scheme, record_file_path, record_stream, is_executable=False
2 ~\poetry\venv\lib\site-packages\installer\utils.py:211 in construct_record_file
209| writer = csv.writer(stream, delimiter=",", quotechar='"', lineterminator="\n")
210| for scheme, record in records:
> 211| writer.writerow(record.to_row(prefix_for_scheme(scheme)))
212| stream.seek(0)
213| return stream.detach()
1 ~\poetry\venv\lib\site-packages\installer\destinations.py:271 in prefix_for_scheme
269| if file_scheme == scheme:
270| return None
> 271| path = os.path.relpath(
272| self.scheme_dict[file_scheme],
273| start=self.scheme_dict[scheme],
ValueError
path is on mount 'C:', start on mount 'D:'
at C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\ntpath.py:703 in relpath
699| path_abs = abspath(normpath(path))
700| start_drive, start_rest = splitdrive(start_abs)
701| path_drive, path_rest = splitdrive(path_abs)
702| if normcase(start_drive) != normcase(path_drive):
> 703| raise ValueError("path is on mount %r, start on mount %r" % (
704| path_drive, start_drive))
705|
706| start_list = [x for x in start_rest.split(sep) if x]
707| path_list = [x for x in path_rest.split(sep) if x]
Cannot install dunamai.
In the past several other projects hit the issue as well, which was solved, e.g. here https://github.com/Lightning-AI/pytorch-lightning/pull/16164/files, by special-casing os.relpath on windows.
We are having the same problem with our Windows runners on Github Actions while attempting to upgrade to Poetry 2.x.
The plugin we are trying to use is poetry-dynamic-versioning. We're using the latest version of poetry (2.1.1) and poetry-dynamic-versioning (1.7.1). We have two different configurations, running python 3.9 and 3.12 respectively. Same error in each case.
Same error message that others have reported:
Installing C:\Users\runneradmin\miniconda3\envs\pxt\Scripts\dunamai.exe over existing file
ValueError
path is on mount 'C:', start on mount 'D:'
at ~\miniconda3\envs\pxt\lib\ntpath.py:703 in relpath
699| path_abs = abspath(normpath(path))
700| start_drive, start_rest = splitdrive(start_abs)
701| path_drive, path_rest = splitdrive(path_abs)
702| if normcase(start_drive) != normcase(path_drive):
> 703| raise ValueError("path is on mount %r, start on mount %r" % (
704| path_drive, start_drive))
705|
706| start_list = [x for x in start_rest.split(sep) if x]
707| path_list = [x for x in path_rest.split(sep) if x]
Cannot install dunamai.
Please advise - what is the officially suggested solution / workaround? It is not clear to me from the thread on this ticket. This is blocking our migration to Poetry 2.x.
Thank you! Aaron
what is the officially suggested solution / workaround?
Especially in CI, you can work around this issue by just installing the required plugins explicitly instead of relying on requires-plugins. (If the plugins are already installed, requires-plugins only verifies that they are installed.) The exact method depends on how you installed Poetry, see https://python-poetry.org/docs/plugins/#using-plugins
Well that worked. Thank you! But it makes me kind of sad, because proper management of plugin dependencies is one of the improvements in Poetry 2.x that I was excited about.
I hope that the poetry team is able to prioritize a fix for this.
Thanks again for your help, Aaron
Especially in CI, you can work around this issue by just installing the required plugins explicitly instead of relying on
requires-plugins. (If the plugins are already installed,requires-pluginsonly verifies that they are installed.) The exact method depends on how you installed Poetry, see https://python-poetry.org/docs/plugins/#using-plugins
what is the officially suggested solution / workaround?
Especially in CI, you can work around this issue by just installing the required plugins explicitly instead of relying on
requires-plugins. (If the plugins are already installed,requires-pluginsonly verifies that they are installed.) The exact method depends on how you installed Poetry, see https://python-poetry.org/docs/plugins/#using-plugins
Naive attempt w/ -run: pip install poetry-dynamic-versioning does not work:
https://github.com/intercreate/smpmgr/actions/runs/13887524548/job/38854235997?pr=48
Reference from https://python-poetry.org/docs/plugins/#using-plugins, - run: pipx inject poetry poetry-dynamic-versioning does work:
https://github.com/intercreate/smpmgr/actions/runs/13887563282/job/38854321439
Same issue here: https://github.com/fcole90/python-poetry-starter/actions/runs/14823882978/job/41614514354 Repo: https://github.com/fcole90/python-poetry-starter
Thanks in advance to anybody looking into this 🙏
The same issue appears to break all the work with poetry (as it obviously begins with running install command) in case the project is located on any drive other than drive C:. Are there any plans to resolve this?
Is there any update on this issue? It's been a known issue for 5 months now and entails some fragile/undesirable workarounds. Is there anything we can do to help?
It's been a known issue for 5 months now and entails some fragile/undesirable workarounds.
In my opinion, the workarounds are pretty decent. Before Poetry 2 there was no requires-plugins, so that the workarounds of today were the standard workflow of that time.
Is there anything we can do to help?
You can provide a PR with a fix. (Most of our regular contributors do not have much time at the moment. We all donate our spare time.)
You can provide a PR with a fix. (Most of our regular contributors do not have much time at the moment. We all donate our spare time.)
That is very understandable. We would be glad to help, however it would be really important to outline the final requirements for the fix.
Probably, will have to check the impacted code first, I also would like to know if there are other places where we might be exposed to this issue.
When I first opened the issue, it seemed to me that just fixing this bug would not be enough, and some greater revision of the whole codebase is required (please, correct me if I'm wrong). If you could at least roughly formalize the acceptance criteria, we would try our best to commit.
I read through the comments and think with the current information, it is difficult to formalize concrete acceptance criteria other than "it works". The most useful information about the issue seems to be in https://github.com/python-poetry/poetry/issues/10028#issuecomment-2600905940. (Links to action runs in other comments have expired.) There, you can see that the final failure is in installer, not in Poetry itself, and it seems to have to do something with the scheme_dict. Thus, either the fix has to be in installer or we have to provide different paths to installer. It might help if someone could investigate what the scheme_dict looks like when this issure occurs.
Not sure if relevant but in my CI run failure, I notice that poetry seem to be using the wrong version of python. I use actions/setup-python@v5 using their recommended setup, but something is working incorrectly: as poetry needs to be installed in advance with pipx, it seems that following invocations are still using the environment version (3.9) instead of the requested one (3.12) 🤔