pip
pip copied to clipboard
pypa
pip installs *.pyc files with stage directory path in them
Description
I need to install a python package into a stage directory.
The command I run from the source directory is:
pip install {my-pkg-name} --isolated --root=/tmp/stage/dir --prefix=/usr/local --ignore-installed --no-deps
This command installs the package into the right location, but *.cpython-39.pyc files have full paths to *.py files in the source directory.
If paths of *.py are needed pip should adjust them so that they should begin with prefix (/usr/local). This is because the source directory will be deleted before the package is run.
What is wrong in the above command?
Expected behavior
No response
pip version
22.1.2
Python version
3.9
OS
FreeBSD 13.1
How to Reproduce
git clone https://github.com/openqasm/openqasm-pygments.git cd openqasm-pygments pip install openqasm-pygments --isolated --root=/tmp/openqasm-pygments --prefix=/usr/local --ignore-installed --no-deps
Output
No response
Code of Conduct
- [X] I agree to follow the PSF Code of Conduct.
I am having trouble reproducing this actually, after misreading the issue. The .pyc's generated for me point to the root dir specified in the command line.
OS: PopOS 22.04
.pyc files don't contain the source directory for you? Maybe this is a FreeBSD issue.
It contains the /tmp/openqasm-pygments path
This is also a wrong behavior.
/tmp/openqasm-pygments is a stage directory and it shouldn't appear in the installed files.
The paths should begin with /usr/local, the prefix.
I am sorry, I misstated the problem in the original report. The problem is that the stage directory appears in *.pyc files (not the source directory).
Why does this matter? The format and content of bytecode files isn't (as far as I know) meant to be directly used by the user, so what user-visible impact does having the directory where the file was compiled in it cause?
Stage directory is transient and it doesn't exist during the run-time. If it was encoded into the file then at some point this invalid value would be used. The FreeBSD ports framework has an explicit check for that. This check caught this problem.
If it was encoded into the file then at some point this invalid value would be used.
In what way? That's what I'm trying to understand. We're adding complexity to pip, but there's no evidence that the thing we're fixing will actually cause a problem.
The FreeBSD ports framework has an explicit check for that. This check caught this problem.
Do their checks explain how this can cause an issue?
In what way? That's what I'm trying to understand. We're adding complexity to pip, but there's no evidence that the thing we're fixing will actually cause a problem.
It is presumed that if the path is there it would be used for something. Otherwise why does pip even write it there? Why don't you change pip to remove this field? This would be fine.
Do their checks explain how this can cause an issue?
This check doesn't fail even for a single port out of 33,000. I also agree that this is a valid test.
It is presumed that if the path is there it would be used for something
I don't think that's necessarily a valid assumption. It could be purely informational. And in any case, it's put there by the stdlib compile mechanism, not by pip, so you'd need to ask the Python developers what it's used for, not the pip developers.
Otherwise why does pip even write it there?
Because we use the stdlib compileall function, which writes a filename. We don't care, all we want is to get a valid .pyc file.
Why don't you change pip to remove this field?
Because pip isn't writing it. Why doesn't CPython change the stdlib compileall function to remove this field? I din't know. You'd have to ask them.
Basically, I'm -1 on changing pip here unless someone can demonstrate a situation where the current behaviour actually causes an issue. We'd need that evidence anyway, as I'd expect any change to include a test, and we can't test whether the "problem" has been fixed unless we can reproduce a problem that's being caused by this.
This problem doesn't happen when distutils are used to compile and install python modules.
Just to add some background: I am trying to modernize handling of Python ports in the FreeBSD ports framework. Currently it isn't able to build Python projects that don't contain setup.py It appears that pip is the right way to go (please correct me if I am wrong). But this issue with stage directories being in *.pyc files prevents me from proceeding.
