panda3d
panda3d copied to clipboard
panda3d
Python source packages do not have included data copied during application build
Description
When specifying python source packages in the requirements.txt, included data that is part of the built wheel is excluded from the final application build.
In my specific case I'm using the master branch of simplepbr to pick up a few fixes and enhancements, since there hasn't been a recent release including them, but the renderer requires shader files, which end up being excluded during the final application build. Note that this is made more confusing due to the line:
Copying data files for module: simplepbr
being in log output, despite that there are no files copied.
This can be worked around by building the wheels as a separate step and including the local build in the requirements file, but this means that packages end up being specified twice, and you can't simply pip install -r requirements.txt for the dev environment.
Steps to Reproduce
$ python repro_setup.py build_apps
$ ./build/manylinux2014_x86_64/repro
$ cat ~/.local/share/repro/output.log
Known pipe types:
eglGraphicsPipe
(all display modules loaded.)
Traceback (most recent call last):
File "__main__", line 27, in <module>
File "__main__", line 23, in repro
File "__main__", line 11, in __init__
File "simplepbr", line 344, in init
File "simplepbr", line 122, in __init__
File "simplepbr", line 206, in _recompile_pbr
File "simplepbr", line 45, in _load_shader_str
FileNotFoundError: [Errno 2] No such file or directory: '/panda_dev/build/manylinux2014_x86_64/shaders/simplepbr.vert'
Environment
- Operating system: Ubuntu 20.04
- Docker Container base: nvidia/opengl:1.2-glvnd-runtime-ubuntu20.04
- System architecture: x86_64
- Panda3D version: 1.10.9
- Installation method: pip
- Python version: 3.8.5
deploy-ng does not support building from source (it would be difficult to support this). I am surprised that it is not failing with an error message.
We are already passing the flag telling pip not to download source distributions, but apparently that gets overridden by the fact that you specified an explicit URL to one.
I do not know how to (elegantly) prevent pip from downloading an explicitly specified source archive or detect that this has happened.
simple-pbr has since updated their package, so this is currently not an issue for me. I've tweaked some other aspects of my build process, too, so would now likely just upload a built wheel to a private package repository (rendering this moot altogether).
Keep this open for other similar cases, or close as no longer relevant?
I think the solution to this is either (1) detecting that a non-wheel has been downloaded into the wheel cache and then exiting with an error, or (2) detecting that a direct link to a source tarball (instead of a wheel) has been placed into the requirements.txt file and then exiting with an error.
I vote for option 1 as it would be pretty trivial to do as part of download_wheels().
