openmc icon indicating copy to clipboard operation
openmc copied to clipboard

Python wheels

Open akoen opened this issue 1 year ago • 6 comments

Description

Adds a CI step to build python wheels with cibuildwheel. Currently only builds for CPython 3.11 but can be expanded to encompass 3.7--3.12 with a one-line change.

Discussion at: https://openmc.discourse.group/t/alpha-python-wheels-for-openmc-including-apple-sillicon/3518

Checklist

  • [x] I have performed a self-review of my own code
  • [x] I have run clang-format on any C++ source files (if applicable)
  • [x] I have followed the style guidelines for Python source files (if applicable)
  • [ ] I have made corresponding changes to the documentation (if applicable)
  • [ ] I have added tests that prove my fix is effective or that my feature works (if applicable)

akoen avatar Oct 17 '23 18:10 akoen

I'm sure @hassec will be keen to take a look at this PR

shimwell avatar Oct 17 '23 19:10 shimwell

Thanks @hassec and @paulromano for the feedback! I hadn't realized that others had made similar efforts. A few thoughts:

  • This is very much a proof-of-concept. My main goal was just to build wheels for macos arm64 since some of my colleagues were having trouble building from source. Hence the big focus on cross-compilation.
  • I deliberately avoided building openmc itself in setup.py for a few reasons.
    1. Those with an existing system-wide installation of openmc might not want to install it locally just to install the python package in a new virtual environment.
    2. When cibuildwheel builds wheels for many python versions in a single workflow, it's much faster to only build openmc once instead of for each python version
    3. I think it makes sense to include DAGMC by default in a python wheel, and that + MOAB must be installed independently regardless.
  • scikit-build is likely the cleanest way to do this, but I had trouble getting cross-compilation working and got the setuptools based approach working first.

I'm happy to clean this PR up and address some of this feedback, but a rework with scikit-build is likely not something I'll have time for since I'm nearing the end of my internship.

akoen avatar Oct 19 '23 01:10 akoen

Had a watch on an issue (https://github.com/pypa/auditwheel/issues/340) over at auditwheel that is relevant to how packaging works for e.g. the openmc executable. So just wanted to leave an FYI here that this was recently fixed via https://github.com/pypa/auditwheel/pull/443 in case it's helpful once this or a follow up PR gets going again

hassec avatar Feb 05 '24 14:02 hassec

  • scikit-build is likely the cleanest way to do this, but I had trouble getting cross-compilation working and got the setuptools based approach working first.

Could you expand on this? Were not able to get it work on all environments that you asked for, or are you trying to run on one platform and build for another? If it's the latter I think just throwing different GHA runners at it should be good enough.

MicahGale avatar Mar 28 '24 17:03 MicahGale

FYI: I was looking into how lxml builds wheels since they use a compiled backend and just seem to always work. It turns out they produce around 20+ wheels.

Looking at their GHA it is actually pretty straightfoward-ish. They most rely on cibuildwheel, which seems to do most of the work for them.

MicahGale avatar Jul 03 '24 20:07 MicahGale

Great to catch up at the meshing group @akoen

I just wanted to make note of something paulromano found.

This pypi package includes Python bindings for the MPI and includes MPICH binaries which is something we had not noticed before. Perhaps the first pypi package to include mpi binaries https://pypi.org/project/mpi4py-mpich/

I notice this nice wheel you have made does not include MPI but perhaps with this package it might be able to

shimwell avatar Jul 09 '24 15:07 shimwell