Add building of pyodide universal wheels

[twiecki]

Closes https://github.com/pymc-devs/pytensor/issues/285 and https://github.com/pymc-devs/pytensor/issues/360.

These require pure-python wheels which does not work if we ship a cython-version of scan, so this build does not include this extension module.

[maresb]

Looks pretty straightforward. Does anyone know offhand it a platform-specific wheel takes precedence over a universal one? I'd hope it does.

[twiecki]

Looks pretty straightforward. Does anyone know offhand it a platform-specific wheel takes precedence over a universal one? I'd hope it does.

Yes, they take precedence. And since we have wheels for all major platforms, the universal one should really only ever be installed when installed through pyodide.

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 81.69%. Comparing base (7fffec6) to head (19cf45e). Report is 93 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #918   +/-   ##
=======================================
  Coverage   81.69%   81.69%           
=======================================
  Files         182      182           
  Lines       47585    47585           
  Branches    11584    11584           
=======================================
  Hits        38875    38875           
- Misses       6518     6520    +2     
+ Partials     2192     2190    -2     

see 2 files with indirect coverage changes

[ricardoV94]

To clarify, did this PR addressed the concerns here: https://github.com/pymc-devs/pytensor/issues/285#issuecomment-1846187274 ?

[maresb]

We can't make pytensor behave as a universal wheel because it has a cython extension.

I believe the idea here is to sidestep the issue by omitting the cython extension. (I don't understand the implications of omitting it.)

[twiecki]

To clarify, did this PR addressed the concerns here: #285 (comment) ?

I think so, yes. Ultimately, having a cython-extension really only makes sense if using the c-backend. For the python backend there shouldn't be any dependency that's related to compilation.

[twiecki]

Thanks for the assist @maresb!

[maresb]

I don't think we're testing the universal dist. Probably we should in order to avoid potentially uploading corrupt wheels.

Also, while trying to determine whether or not we're testing, I see that we're using the sloppy old versions for artifact upload/download, see #560, #580.

[maresb]

@lucianopaz, I'm very interested to get your thoughts on this approach. Is it viable to simply omit the extension when building a universal wheel?

[maresb]

My reading of the stack overflow answer is that the priority is:

platform-specific compatible wheel > universal wheel > sdist

[twiecki]

As pointed out here, universal wheels will be preferred by pip over platform specific wheels.

I don't think that's the case. And I have confirmed that non-pyodide installs work the same across all major platforms.

I don't think the risk is that high to give this a try. We can test to make sure we don't break anything on existing platforms.

[lucianopaz]

My reading of the stack overflow answer is that the priority is:

platform-specific compatible wheel > universal wheel > sdist

Oh! You’re right, I misread the last sentence

This is not all that common however, as a universal wheel would be preferred over a source distribution!

So platform specific wheels will be preferred, then universal wheels and finally source distributions. I take back what I said about the risks then. This should be mostly fine.

[twiecki]

Thanks @lucianopaz. Can we get an approval?

[maresb]

I'm a bit uncomfortable that we're not testing the universal wheel. Maybe I can finish off #560 right now...

[twiecki]

FAILED tests/test_config.py::test_config_pickling - AssertionError: Regex pattern did not match.
 Regex: "Can't pickle local object"
 Input: "Can't get local object 'test_config_pickling.<locals>.<lambda>'"
= 1 failed, 807 passed, 69 skipped, 4 xfailed, 33 warnings in 248.68s (0:04:08) =
Error: Process completed with exit code 1.

what might this be about?