Pure wheel too

[henryiii]

trafficstars

Apologies if covered elsewhere, but it seems to be that wrapt has optional extensions. If that's the case, it should ship the pure Python wheel too, pip should select the most specific wheel, IIRC. This currently keeps libraries using wrapt from being installed into pyodide. I can request wrapt be added to the custom wheelhouse for pyodide, but this would be a simpler fix, as pyodide supports pure Python wheels. It also likely would be useful for Pythonista on iOS and other non-traditional Python platforms.

Would it be helpful to build that optional extension, though, in enscriptem for Pyodide? If that's the case, requesting it be build there might still be helpful.

Example request (for copy and paste)

🐍 Package Request

• wrapt / latest
• https://pypi.org/project/wrapt/
• Package Dependencies that needs to be resolved first: None.

Note that https://github.com/GrahamDumpleton/wrapt/blob/develop/setup.py tries to disable pure Python extensions on some platforms, I'd probably try building them first before letting them be disabled?

FYI, pyodide is now embedded into the middle of the numpy.org page.

[GrahamDumpleton]

This is the ultimately the fault of the Python packaging system.

Prior to wheels, when installing wrapt if the compilation of the C extension failed the setup.py would through some magic fallback to installing the pure Python version. The Python packaging system mechanisms one has to use now to support wheels doesn't allow that anymore and if C compilation fails, it fails outright, it isn't possible to have it fallback to using a pure Python installation if that is possible.

If a pure Python source code wheel were provided then it would deny, or make it harder, for any system for which binary wheels aren't provided from installing wrapt with the C extension, which would be a worse default case.

So sort of limited now by how the latest Python packaging system works and there doesn't look to be a good solution.

Easiest thing might be that if you work out a check that could be done so it can be detected that installing under special case of pyodide, that can be added at:

• https://github.com/GrahamDumpleton/wrapt/blob/develop/setup.py#L18

and the extension forcibly disabled. Otherwise the only way is to set WRAPT_INSTALL_EXTENSIONS=false environment variable when building/installing wrapt.

Only other option can think of is to try and abuse the extra_requires feature of Python packaging so that if installed something like wrapt[disable-extension] it would force pure Python version. Am not sure if this would help though as would need to ensure that is installed first somehow since other stuff would still reference it as wrapt so dependent on ordering.

[henryiii]

Pyodide cannot install SDists. It looks for pure wheels only. Adding custom code will not help. For this specific case (not for general or future cases), I can request it be included.

How horrible is it[^1] to use the pure python version if a wheel isn't provided? We can cover a really, really wide range of wheels today (cibuildwheel can make 83, not counting stable API ones), and if none of those match, is it really better to try to compile on the user's machine if you could provide a Python fallback? Those platforms don't get built wheels for anything anyway. You can always pip install --no-binary=wrapt ... (I believe) to get the build to happen. I guess it depends on how bad the pure Python is vs. the built extension, but it would have to be pretty bad.

I don't think it's a Python packaging system problem, but a setuptools one. There is a lot of work in the ecosystem to provide alternatives to setuptools, and we are just beginning to get into the era of having reasonable ones that can build (meson in early stages, scikit-build in even earlier stages, at least as far as proper PEP 517 builds go).

[^1]: I have no idea how bad it is for wrapt, maybe it's terrible and doesn't work in pure Python form? Takes 2 seconds to load? Etc...

[GrahamDumpleton]

When I mean Python packaging I specifically mean setuptools. In distutils I could implement the fallback to pure Python, but not in setuptools.

For performance differences see:

• https://wrapt.readthedocs.io/en/latest/benchmarks.html

[GrahamDumpleton]

BTW, does pyodide have its own specific platform designator for Python wheels. So if PyPi hosted something like wrapt-1.13.3-whatpyodieuses.whl which was a source wheel, would it use it. Would be sort of handy if it did.

[mentalisttraceur]

@GrahamDumpleton re: "does pyodide have its own specific platform designator for Python wheels" - per PEP-425 platform tag rules, it ought to be emscripten_1_0_wasm32:

Welcome to the Pyodide terminal emulator 🐍
Python 3.10.2 (main, Apr  9 2022 20:52:01) on WebAssembly VM
Type "help", "copyright", "credits" or "license" for more information.
>>> import setuptools
>>> import distutils.util
>>> 
>>> distutils.util.get_platform()
'emscripten-1.0-wasm32'
>>> 
>>> distutils.util.get_platform().replace('.', '_').replace('-', '_')
'emscripten_1_0_wasm32'

[GrahamDumpleton]

Not sure if that helps me at this point. I am assuming I would need a way from cibuildwheel infrastructure I use from GitHub actions to generate a specific wheel for that target. Is it even plausible that cibuildwheel could do that?

[henryiii]

Not yet, and pyodide can only load wheels created built-in, and WASM is not binary stable yet anyway. We do plan to try to make it available, though.

We added wrapt to the pyodide build process, so for pyodide it’s fine now. Other WASM builds of Python (like CPython 3.11) still would likely benefit from a pure Python fallback. Maybe. Not sure what pip would look like with no shell available (pyodide rewrite it as micropip).

[GrahamDumpleton]

I guess I probably just need to do that distinct from cibuildwheel parts of build following something like:

• https://stackoverflow.com/questions/66334450/how-to-build-py3-none-any-wheels-for-a-project-with-an-optional-c-extension

I'll see if I can some time to play with it over the next week.

[GrahamDumpleton]

Are you able to verify if wrapt version 1.15.0rc1 satisfies what you need? There is a:

wrapt-1.15.0rc1-py3-none-any.whl

file as well as:

wrapt-1.15.0rc1.tar.gz

and all the platform specific variants.

[mentalisttraceur]

Well, a quick manual install+import test of 1.15.0rc1 on https://pyodide.org/en/stable/console.html works:

>>> import micropip
>>> await micropip.install('https://files.pythonhosted.org/packages/95/06/7626d5c52dda7357aee9c546cd099485fe62b4c97bcf97ea0dd177317cd0/wrapt-1.15.0rc1-py3-none-any.whl')
>>> import wrapt
>>> wrapt.__version__
'1.15.0rc1'
>>> 

[henryiii]

I was planning on testing this with the sort-of official WASM Python builds. Not near a computer for a while But it should work and sounds great. :)

[mentalisttraceur]

The above test with explicit URL doesn't verify that the right wheel will get selected, but since Pyodide now bundles a vendored copy of wrapt it doesn't seem to look for it in PyPI anymore if you just give it the package name.

But I'm pretty confident it would select the right wheel once they stop vendoring, because I touched that code recently.

[GrahamDumpleton]

Going to close this at this point as believe version 1.15.0 addresses need for pure Python code wheel.