ANTsX
PyPI updating strategy
As discussed in #578 and #580
From @ncullen93:
looks like it's not the specific build size but just the total storage on pypi being over 10gb? So some old wheels need to be deleted? For what it's worth, I really don't think you need to build wheels for each python distribution specifically. I think it's more common to just build one wheel for "3.x" - see e.g., the pypi workflow I use elsewhere.
I don't have permission to delete old wheels. I am not attached to building every python, I don't know if there's stats on which versions get downloaded the most but will look into it.
What do others (devs and users) think about building fewer wheels?
@stnava with the bug fixes at 0.5.1 it might be good to update PyPI. Python 3.8 will reach EOL in October 2024, so maybe that can be dropped.
Hopefully we will get more space to play with once the more efficient wheels are available via #591 .
Yes the new wheels will be very small. It should be done within a few weeks.
I think I have to be an owner to change things, yes. Whatever I am now lets me see the project under my project, but I can only view.
@cookpa can we try to build wheels now and see the size? When I build locally using python -m build it's only 16mb but that seems too small.
Windows is failing because it explicitly calls setup.py, let's see if the others build
@ncullen93 I got the Linux wheels to build, the artifacts are named funny but I can fix that. I installed the python 3.10 Linux wheel and it runs sucessfully! Amazing that it's only 17Mb
On Intel Mac, there is a problem with cibuildwheel, it is getting into trouble with the MACOSX_DEPLOYMENT_TARGET. I think it might relate to this known issue with scikit-build
https://github.com/pypa/cibuildwheel/issues/952
Bumping the version of scikit-build and / or cibuildwheel might help
Yeah I see the small size is also replicated on the ci build. Hoping it is for real. I will try to fix the wheel issues. edit: but I will wait to see if your PR fixes it
I'm hoping that setting MACOSX_DEPLOYMENT_TARGET=10.14 in the environment will solve the problem. Somewhere it is setting a default of 10.9 and not respecting the pyproject.toml entry.
I may also have to downgrade cibuildwheel to 2.16.2 again, because the latest version appears to break the package delocalization on arm64.
Wheels are looking good, they fall over occasionally but mostly seem to build. It's been a long time since a release made it to PyPI, it would be good to get a new one up.
To do:
-
Make space on PyPi. They discourage deleting releases, but it seems like we've done it before, and probably have no choice but to delete at least 0.3.8. The new wheels are so much smaller, we can get 10-20 releases for the same storage, more if we don't support every python version.
-
Tag new release. This should trigger the upload to PyPI, but I might need some help if it fails (ie, to manually put some uploads up)
One option would be to leave one wheel per platform of 0.3.8 for now (which?)
Most recent pypi stats. Mostly Linux, less older python now. I could clean up some 0.3.8 packages and try a new release if it's OK with everyone
Sure, thanks Phil.On Aug 5, 2024, at 6:56 PM, Philip Cook @.***> wrote: image.png (view on web) image.png (view on web) Most recent pypi stats. Mostly Linux, less older python now. I could clean up some 0.3.8 packages and try a new release if it's OK with everyone
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>
Deleted 0.3.8, figure we may as well make more space and not deal with this again for longer.
I'm ready to tag v0.5.3. Do I need to do anything else before releasing? I presume after I tag, I should edit pyproject.toml and ants/__init__.py, and increment the version to 0.5.4?
It worked! But the workflow only sends Linux wheels. Is there some other mechanism to upload the other platforms?
yes - it's via twine --- this is from the ANTsPyMM README:
rm -r -f build/ antspymm.egg-info/ dist/
python3 -m build .
python3 -m pip install --upgrade twine
python3 -m twine upload --repository antspymm dist/*
you need to add some stuff to pypirc ie a secret token created on pypi .... I automated this for antspymm which uploads upon a new tag.
I uploaded a source dist, which works, and a bunch of wheels from the Github build. Annoyingly, pip won't install the wheel for my Mac unless I do
SYSTEM_VERSION_COMPAT=0 pip install antspyx
This has long been the case for me with Github wheels. I think I tried to fix this before but will take another look. Hopefully we can get wheels building correctly for the next release
I was able to make a compatible wheel with
export MACOSX_DEPLOYMENT_TARGET=10.14
python -m build --wheel
For some reason, my Intel Mac only lists 10_X as compatible targets, even though I have Mac OS 14. We might therefore want to change the wheels to target an earlier version, currently they target the runner OS (12 for Intel, 14 for ARM).
My next to-do will be to try building wheels for an earlier OS target.
#697 I documented these Mac issues and set the wheel workflow to push everything to PyPI.
I cannot get the GHA workflow to allow an earlier MACOSX_DEPLOYMENT_TARGET, it fails with an error message about libpng being targeted for the current OS. I also have Homebrew libpng, so I'm not sure why my machine is OK with the MACOSX_DEPLOYMENT_TARGET=10.14.
I only added a python 3.11 wheel by hand, but can add others if needed.