``poetry new`` fails when creating a new system environment

[yngvem]

• [x] I am on the latest Poetry version.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Windows 10 Version 21H1 (OS Build 19043.1415)
• Poetry version: 1.1.12
• Python version: Python 3.9.9 (installed via Windows store)
• Link of a Gist with the contents of your pyproject.toml file: N/A

Issue

When I run poetry new {name}, i get the following OSError (full traceback from poetry new {name} -vvv here:

  OSError

  [WinError 1920] The file cannot be accessed by the system: 'C:\\Users\\yngve.moe\\AppData\\Local\\Microsoft\\WindowsApps\\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\\python.exe\\Scripts'

  at C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.2544.0_x64__qbz5n2kfra8p0\lib\pathlib.py:1232 in stat
      1228│
      1229│         Return the result of the stat() system call on this path, like
      1230│         os.stat() does.
      1231│         """
    → 1232│         return self._accessor.stat(self)
      1233│
      1234│     def owner(self):
      1235│
      1236│         Return the login name of the file owner.

I've done some debugging, and it seems like the bug is due to the following lines (at time of writing, commit e34277e): This line creates a new system environment with the path to the python.exe executable (C:\\Users\\yngve.moe\\AppData\\Local\\Microsoft\\WindowsApps\\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\\python.exe) https://github.com/python-poetry/poetry/blob/e34277e24026d80bc88898a5710183383686962f/src/poetry/console/commands/new.py#L66

Then, the self._bin_dir is set to the path to the Python executable (not its parent directory) https://github.com/python-poetry/poetry/blob/e34277e24026d80bc88898a5710183383686962f/src/poetry/utils/env.py#L1082

The next problem arises during this function call: https://github.com/python-poetry/poetry/blob/e34277e24026d80bc88898a5710183383686962f/src/poetry/utils/env.py#L1087

Specifically, it happens on this line, since self._bin_dir.glob is the path to the Python executable, not its parent directory. https://github.com/python-poetry/poetry/blob/e34277e24026d80bc88898a5710183383686962f/src/poetry/utils/env.py#L1139

I am able to circumvent this problem locally by modifying https://github.com/python-poetry/poetry/blob/e34277e24026d80bc88898a5710183383686962f/src/poetry/console/commands/new.py#L66 to be

    current_env = SystemEnv(Path(sys.executable).parent) 

---

EDIT: I am relatively certain that there is something with my Windows setup that leads to this problem, since I have been able to get Poetry to work with Miniconda (which I have uninstalled now) on the same computer before, but I'll leave this issue open since it should work with the installation from Windows store too.

[yngvem]

I reinstalled poetry and now everything works. I'm not sure why.

Edit: I tried again and the issue occured again, so I am unsure if it actually worked before or not.

[xiefanggg]

I'm having the exact same issue

OS version and name: Windows 11 Version 21H2 Poetry version: 1.1.12 Python version: Python 3.10.2 (installed via Windows store)

[liudonghua123]

I have the same issue.

D:\code\python>poetry new poetry-demo

  OSError

  [WinError 1920] The file cannot be accessed by the system: 'C:\\Users\\Liu.D.H\\AppData\\Local\\Microsoft\\WindowsApps\\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\\python.exe\\Scripts'

  at C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.752.0_x64__qbz5n2kfra8p0\lib\pathlib.py:1095 in stat
      1091│
      1092│         Return the result of the stat() system call on this path, like
      1093│         os.stat() does.
      1094│         """
    → 1095│         return self._accessor.stat(self, follow_symlinks=follow_symlinks)
      1096│
      1097│     def owner(self):
      1098│
      1099│         Return the login name of the file owner.

D:\code\python>poetry --version
Poetry version 1.1.13

D:\code\python>where poetry
C:\Users\Liu.D.H\.poetry\bin\poetry
C:\Users\Liu.D.H\.poetry\bin\poetry.bat

D:\code\python>where python
C:\Users\Liu.D.H\AppData\Local\Microsoft\WindowsApps\python.exe
C:\Users\Liu.D.H\anaconda3\python.exe

D:\code\python>ver

Microsoft Windows [Version 10.0.22000.527]

D:\code\python>

[liudonghua123]

After some debugging. I find the reason is that pathlib.Path of ms_python_path throws OSError while official_python_path return expected False.

C:\Users\Liu.D.H>python
Python 3.10.2 (tags/v3.10.2:a58ebcc, Jan 17 2022, 14:12:15) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from pathlib import Path
>>> ms_python_path=Path('C:/Users/Liu.D.H/AppData/Local/Microsoft/WindowsApps/PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0/python.exe/Scripts')
>>> ms_python_path.is_dir()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\pathlib.py", line 1303, in is_dir
    return S_ISDIR(self.stat().st_mode)
  File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\pathlib.py", line 1095, in stat
    return self._accessor.stat(self, follow_symlinks=follow_symlinks)
OSError: [WinError 1920] The file cannot be accessed by the system: 'C:\\Users\\Liu.D.H\\AppData\\Local\\Microsoft\\WindowsApps\\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\\python.exe\\Scripts'
>>> official_python_path=Path(r'C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\python.exe\Scripts')
>>> official_python_path.is_dir()
False
>>>

The related code is in https://github.com/python/cpython/blob/v3.10.2/Lib/pathlib.py#L405, https://github.com/python/cpython/blob/v3.10.2/Lib/pathlib.py#L1303, https://github.com/python/cpython/blob/v3.10.2/Lib/pathlib.py#L1095, https://github.com/python-poetry/poetry/blob/1.1.13/poetry/utils/env.py#L929-L935, https://github.com/python-poetry/poetry/blob/1.1.13/poetry/utils/env.py#L1092.

C:\Users\Liu.D.H\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0 is a special directory which contains a lot zero sized python file, maybe called App execution Alias.

C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310 is an ordinary python installation folder structure.

If I use C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.752.0_x64__qbz5n2kfra8p0 instead of C:\Users\Liu.D.H\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0, maybe it will works. But I have encountered another problem.

[Masoud-Ghodrati]

I have exactly the same issue

OS version and name: Windows 11 home Version 21H2 Poetry version: 1.1.13 Python version: Python 3.10.2 (installed via Windows store)

any solution so far

[liudonghua123]

@Masoud-Ghodrati You can try my modification in the above pr.

[Masoud-Ghodrati]

@Masoud-Ghodrati You can try my modification in the above pr.

no it didn't work, same issue as yours

[xiefanggg]

I uninstalled Python from Windows store, and reinstalled it via the installer I downloaded from https://www.python.org/downloads/release/python-3102/, then the issue resolved. I suspect it has something with the Windows store version of Python.

[Masoud-Ghodrati]

I uninstalled Python from Windows store, and reinstalled it via the installer I downloaded from https://www.python.org/downloads/release/python-3102/, then the issue resolved. I suspect it has something with the Windows store version of Python.

thanks, seems it solved the issue.

[elpablete]

How did you manage to uninstall Python from Windows store?

[Masoud-Ghodrati]

Apps > Apps and Features

[GotFlojo]

Same problem here, with poetry init This generally seems to happen when you install python via the Windows Store installer. In my case installing with the python.org installer is a little tricky because I'm not an adminstrator on the machine in question

poetry/src/poetry/utils/env.py I think it is because SystemEnv calls the Env constructor wich appends /Scripts to the path it gets via sys.executable on Windows. The problem is the resulting path just does not exist. There is no Scripts directory.

This seems to work for the python.org installer but the Store installer seems to layout the directory structure a little differently.

[rbtcollins]

This is due to the Windows Store version of Python not matching some expectations of the poetry code. I haven't looked at that code closely yet, but to help devs analyze this I'm including a backtrace.

backtrace

Breakpoint 1 at c:\program files\windowsapps\pythonsoftwarefoundation.python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\pathlib.py:1198
(Pdb) r
The program exited via sys.exit(). Exit status: 1
> c:\users\robertc\appdata\local\packages\pythonsoftwarefoundation.python.3.8_qbz5n2kfra8p0\localcache\local-packages\python38\site-packages\poetry\__main__.py(1)<module>()
-> import sys
(Pdb) c
> c:\program files\windowsapps\pythonsoftwarefoundation.python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\pathlib.py(1198)stat()
-> return self._accessor.stat(self)
(Pdb) c

This command will guide you through creating your pyproject.toml config.

Package name [dyson-planner]:
Version [0.1.0]:
Description []:
Author [Robert Collins <[email protected]>, n to skip]:
License []:
> c:\program files\windowsapps\pythonsoftwarefoundation.python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\pathlib.py(1198)stat()
-> return self._accessor.stat(self)
(Pdb) bt
  c:\program files\windowsapps\pythonsoftwarefoundation.python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\bdb.py(580)run()
-> exec(cmd, globals, locals)
  c:\users\robertc\appdata\local\packages\pythonsoftwarefoundation.python.3.8_qbz5n2kfra8p0\localcache\local-packages\python38\site-packages\poetry\__main__.py(7)<module>()
-> sys.exit(main())
  c:\users\robertc\appdata\local\packages\pythonsoftwarefoundation.python.3.8_qbz5n2kfra8p0\localcache\local-packages\python38\site-packages\poetry\console\__init__.py(5)main()
-> return Application().run()
  c:\users\robertc\appdata\local\packages\pythonsoftwarefoundation.python.3.8_qbz5n2kfra8p0\localcache\local-packages\python38\site-packages\clikit\console_application.py(131)run()
-> status_code = command.handle(parsed_args, io)
  c:\users\robertc\appdata\local\packages\pythonsoftwarefoundation.python.3.8_qbz5n2kfra8p0\localcache\local-packages\python38\site-packages\clikit\api\command\command.py(120)handle()
-> status_code = self._do_handle(args, io)
  c:\users\robertc\appdata\local\packages\pythonsoftwarefoundation.python.3.8_qbz5n2kfra8p0\localcache\local-packages\python38\site-packages\clikit\api\command\command.py(171)_do_handle()
-> return getattr(handler, handler_method)(args, io, self)
  c:\users\robertc\appdata\local\packages\pythonsoftwarefoundation.python.3.8_qbz5n2kfra8p0\localcache\local-packages\python38\site-packages\cleo\commands\command.py(92)wrap_handle()
-> return self.handle()
  c:\users\robertc\appdata\local\packages\pythonsoftwarefoundation.python.3.8_qbz5n2kfra8p0\localcache\local-packages\python38\site-packages\poetry\console\commands\init.py(144)handle()
-> current_env = SystemEnv(Path(sys.executable))
  c:\users\robertc\appdata\local\packages\pythonsoftwarefoundation.python.3.8_qbz5n2kfra8p0\localcache\local-packages\python38\site-packages\poetry\utils\env.py(942)__init__()
-> self.find_executables()
  c:\users\robertc\appdata\local\packages\pythonsoftwarefoundation.python.3.8_qbz5n2kfra8p0\localcache\local-packages\python38\site-packages\poetry\utils\env.py(1116)find_executables()
-> self._find_python_executable()
  c:\users\robertc\appdata\local\packages\pythonsoftwarefoundation.python.3.8_qbz5n2kfra8p0\localcache\local-packages\python38\site-packages\poetry\utils\env.py(1090)_find_python_executable()
-> python_executables = sorted(
  c:\users\robertc\appdata\local\packages\pythonsoftwarefoundation.python.3.8_qbz5n2kfra8p0\localcache\local-packages\python38\site-packages\poetry\utils\env.py(1090)<genexpr>()
-> python_executables = sorted(
  c:\program files\windowsapps\pythonsoftwarefoundation.python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\pathlib.py(1140)glob()
-> for p in selector.select_from(self):
  c:\program files\windowsapps\pythonsoftwarefoundation.python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\pathlib.py(499)select_from()
-> if not is_dir(parent_path):
  c:\program files\windowsapps\pythonsoftwarefoundation.python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\pathlib.py(1422)is_dir()
-> return S_ISDIR(self.stat().st_mode)
> c:\program files\windowsapps\pythonsoftwarefoundation.python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\pathlib.py(1198)stat()
-> return self._accessor.stat(self)
(Pdb) c

  OSError

  [WinError 1920] The file cannot be accessed by the system: 'C:\\Users\\robertc\\AppData\\Local\\Microsoft\\WindowsApps\\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\\python.exe\\Scripts'

  at C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\pathlib.py:1198 in stat
      1194│
      1195│         Return the result of the stat() system call on this path, like
      1196│         os.stat() does.
      1197│         """
    → 1198│         return self._accessor.stat(self)
      1199│
      1200│     def owner(self):
      1201│
      1202│         Return the login name of the file owner.
The program exited via sys.exit(). Exit status: 1

[henry174Ajou]

I uninstalled Python from Windows store, and reinstalled it via the installer I downloaded from https://www.python.org/downloads/release/python-3102/, then the issue resolved. I suspect it has something with the Windows store version of Python.

Thanks, I solved this issue by uninstalling the Windows store version of Python.