pypa
env: include .path in the IsolatedEnv ABC
This is needed by pyodide, and is already available on the return _IsolatedEnvVenvPip, but not public for typing. Technically pyodide wants the site-packages dir, but it can be computed from the path.
https://github.com/pyodide/pyodide/blob/7a9273dd6c893ab1273f8c5d22f62f60296f13d5/pyodide-build/pyodide_build/pypabuild.py#L27-L28 (Plain .path is available there, but not typed into IsolatedEnv).
Open to other options if this is not a good way to go.
It is unclear to me this is the best approach, would the pyodide maintainers be willing to discuss with us their needs, so that we can all be on the same page and find the best solution?
cc @hoodmane
Yeah. Our needs are specifically that we are trying to do cross-compilation. Certain packages have both build time and run time behavior. If one of these packages is requested at build time we need to provide a specific version of it. There are currently four affected packages: pycparser, cffi, numpy, and scipy.
The simplest examples are pycparser and cffi, which don't require modified cross-build wheels, we just need to match the build version provided to the version that will be available at run time. This could be managed with env.install by just replacing the requirement with a pinned requirement.
Numpy does feature detection based on the target and creates numpy/core/include/numpy/_numpyconfig.h based on it. We also need to replace the libraries numpy/core/lib/libnpymath.a and numpy/random/lib/libnpyrandom.a with wasm target libraries. We create a cross-build numpy by downloading a many-linux wheel and then replacing these files in the wheel with the appropriate wasm target versions.
Similarly, in scipy, downstream libraries use the cython header files linalg/cython_lapack.pxd and scipy/linalg/cython_blas.pxd. We have to adjust these header files in order to make them work and if statsmodels and scikit-learn don't get the adjusted headers, they will crash at runtime.
Perhaps for numpy and scipy we could use env.install with a path to a local wheel?
The new default isolated env exposes its path so I'll go ahead and close this.
Great! @hoodmane, pyodide-build might need to adapt after the next release.