Permission problems in WSL

[espositofulvio]

pyflow verison: 0.1.5 OS: WSL (windows subsystem for linux ver. 1) I started a new project with pyflow new, then tried to add dependencies and getting (run with backtrace=full):

RUST_BACKTRACE=full pyflow install starlette                                                                                                Found lockfile
🔍 Resolving dependencies...
⬇ Installing starlette 0.12.9 ...
thread 'main' panicked at 'Problem reading the dist directory of a package built from source. The `wheel` package have not have been installed in this environment.: Os { code: 2, kind: NotFound, message: "No such file or directory" }', src/libcore/result.rs:1084:5
stack backtrace:
   0:     0x7f782526cceb - <unknown>
   1:     0x7f782526c9c6 - <unknown>
   2:     0x7f782526d45d - std::panicking::rust_panic_with_hook::h92f98b46e22f14ed
   3:     0x7f782526cfe2 - <unknown>
   4:     0x7f782526cec6 - rust_begin_unwind
   5:     0x7f782528ef9d - core::panicking::panic_fmt::h7e9f94035af782b3
   6:     0x7f782528f097 - core::result::unwrap_failed::hf7591c1dd9412006
   7:     0x7f7824f719d4 - <unknown>
   8:     0x7f7824f57652 - <unknown>
   9:     0x7f7824f5fcb0 - <unknown>
  10:     0x7f7824ec7f23 - <unknown>
  11:     0x7f782526ce63 - <unknown>
  12:     0x7f7825276b8a - __rust_maybe_catch_panic
  13:     0x7f782526da2d - std::rt::lang_start_internal::hea1b49a567afe309
  14:     0x7f7824f644b2 - <unknown>
  15:     0x7f7823701b97 - __libc_start_main
  16:     0x7f7824e8c289 - <unknown>
  17:                0x0 - <unknown>

I managed to install yapf, but many others fail. For example installing pytest breaks when wcwidth is installed for example.

[David-OConnor]

Could you please post all of the dependencies this error occured for? (ie a copy+paste of the [pyflow.dependencies] section of pyproject.toml.

[David-OConnor]

Additionally - could you verify that wheel is installed to the venv by opening up __pypackages__/3.x/.venv/lib64/python3.x/site-packages, and checking if there's a folder called wheel? (lib64 may be named lib)

[David-OConnor]

Probably the best way to diagnose: Run this from __pypackages__/3.x/lib/starlette: ../../.venv/bin/python setup.py bdist_wheel

What's the output, and is a dist folder then created in __pypackages__/3.x/lib/starlette?

[espositofulvio]

Could you please post all of the dependencies this error occured for? (ie a copy+paste of the [pyflow.dependencies] section of pyproject.toml.

At the moment I'm just starting, the following is my pyproject.toml

[tool.pyflow.dependencies]
asyncpg = "^0.18.3"
starlette = "^0.12.9"


[tool.pyflow.dev-dependencies]
yapf = "^0.28.0"
pytest = "^5.2.1"

it managed to install asyncpg and yapf fine, but it always fails on both pytest (actually wcwidth, which is a dependency) and starlette.

Additionally - could you verify that wheel is installed to the venv by opening up __pypackages__/3.x/.venv/lib64/python3.x/site-packages, and checking if there's a folder called wheel? (lib64 may be named lib)

yep it's there

Probably the best way to diagnose: Run this from __pypackages__/3.x/lib/starlette: ../../.venv/bin/python setup.py bdist_wheel

What's the output, and is a dist folder then created in __pypackages__/3.x/lib/starlette?

__pypackages__/3.x/lib/starlette doesn't exist. I've just got __pypackages__/3.x/lib/starlette-0.12.9

[David-OConnor]

Thank you. Could you run that command in __pypackages__/3.x/lib/starlette-0.12.9 ? My mistake; the original dir is what the built will be extracted to. The one that's there for you is the source dir it should build the wheel from. I'm attempting to see why the build is failing.

[espositofulvio]

OK, I've run it and got:

running bdist_wheel
running build
running build_py
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
running install_data
running install_egg_info
running egg_info
writing starlette.egg-info/PKG-INFO
writing dependency_links to starlette.egg-info/dependency_links.txt
writing requirements to starlette.egg-info/requires.txt
writing top-level names to starlette.egg-info/top_level.txt
reading manifest file 'starlette.egg-info/SOURCES.txt'
writing manifest file 'starlette.egg-info/SOURCES.txt'
removing 'build/bdist.linux-x86_64/wheel/starlette-0.12.9-py3.7.egg-info' (and everything under it)
Copying starlette.egg-info to build/bdist.linux-x86_64/wheel/starlette-0.12.9-py3.7.egg-info
running install_scripts
error: [('build/bdist.linux-x86_64/wheel/starlette-0.12.9-py3.7.egg-info', 'build/bdist.linux-x86_64/wheel/starlette-0.12.9.dist-info', "[Errno 13] Permission denied: 'build/bdist.linux-x86_64/wheel/starlette-0.12.9.dist-info'")]

[David-OConnor]

Thanks - that's the underlying error preventing the packages from installing. Can you tell me more about the WSL install? Which dist/version? How did you install pyflow? (ie deb? Did you install the deb with sudo? If the standalone bin, where did you put it?) and the directory the proj lives.

[espositofulvio]

It's still WSL 1, it's the laptop I don't mess with :) distro is Ubuntu 18.04, I installed pyflow from the deb file with sudo apt install ./pyflow_0.1.5_amd64.deb. Yep I did install it with sudo, apt refused to work without it. The directory the proj lives in is on the windows side, I try to keep everything readable from Windows as well, so it's in /mnt/d/Work/Personal/projects/catalysis. Hope this is enough, otherwise let me know how can I help you more. And thanks for all your efforts.

[David-OConnor]

I really appreciate all the info! Could you please try with a project dir somewhere on the normal WSL path (eg ~/projname), and see if it works there?

[David-OConnor]

Got another one - is the python being used a system python pyflow found, or one pyflow installed? If a system python, could you run python3 setup.py bdist_wheel in the starlette-0.12.9 folder and post the result?

[espositofulvio]

yep, it works if the directory is inside WSL, I tried ~/projects/catalysis. I can actually have it there so it's non-blocking for me at this point. The python used is the one found by pyflow, I use pyenv and it found one of those. running the python3 command gave an error because wheel wasn't installed system-wide, so I pip-installed it and then ran it again, but same error:

running bdist_wheel
running build
running build_py
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
running install_data
running install_egg_info
running egg_info
writing starlette.egg-info/PKG-INFO
writing dependency_links to starlette.egg-info/dependency_links.txt
writing requirements to starlette.egg-info/requires.txt
writing top-level names to starlette.egg-info/top_level.txt
reading manifest file 'starlette.egg-info/SOURCES.txt'
writing manifest file 'starlette.egg-info/SOURCES.txt'
removing 'build/bdist.linux-x86_64/wheel/starlette-0.12.9-py3.7.egg-info' (and everything under it)
Copying starlette.egg-info to build/bdist.linux-x86_64/wheel/starlette-0.12.9-py3.7.egg-info
running install_scripts
error: [('build/bdist.linux-x86_64/wheel/starlette-0.12.9-py3.7.egg-info', 'build/bdist.linux-x86_64/wheel/starlette-0.12.9.dist-info', "[Errno 13] Permission denied: 'build/bdist.linux-x86_64/wheel/starlette-0.12.9.dist-info'")]

[David-OConnor]

An immediate change this calls for is to improve the error message. It might be possible to solve this directly (I don't have the solution now), but I think the long-term fix will be to find a better way to build wheels from source, ie not rely on wheel and setuptools. (Something to do with pep517?)

[espositofulvio]

At the moment I guess just warning users in the README that if they use WSL 1 (not tested with WSL 2) packages might not install if placed on the NTFS mount would be enough. Something tells me that WSL 2 will solve this, but until I get the update I can't test it.

[David-OConnor]

I'm able to reproduce on every WSL distro I've tried. A quick and dirty fix may be to check if the path is in /mnt, and if so, move the source folder into a temporary directory outside /mnt, then move the wheel back to the proj folder.

[espositofulvio]

I'd love to be more helpful but my knowledge of rust is close to non existent. I wonder what exactly prevent the wheel to be built. Anyway, I can live with these projects being in the wsl filesystem, I would be only curious if anyone could try WSL 2 to see if the issue still persists.

[Mogost]

I'm not 100% sure, but it's probably a very similar error https://github.com/Microsoft/WSL/issues/14 If it’s not, I apologize for bothering

[David-OConnor]

I agree that it looks similar. I haven't made any progress on this.