torch-mlir icon indicating copy to clipboard operation
torch-mlir copied to clipboard

Publish releases to PyPI after build

Open dellis23 opened this issue 3 years ago • 5 comments

dellis23 avatar Oct 04 '22 18:10 dellis23

See https://github.com/llvm/torch-mlir/issues/1445

dellis23 avatar Oct 04 '22 18:10 dellis23

I haven't tested with audit wheel before so hopefully it just works -- but if not we may need to fix to make sure it passes.

Looks like I need to do something else to get it to work:

usage: auditwheel [-h] [-V] [-v] command ...
auditwheel: error: cannot access /wheelhouse/torch_mlir-*-cp39-cp39-linux_x86_64.whl. No such file

dellis23 avatar Oct 05 '22 13:10 dellis23

@powderluv Do you have any tips on testing the release process in a PR? The issue I'm having: I'm doing a full build, but because release_id isn't set, it skips the publishing part. See here. I could just take a risk and merge / iterate on main, but I'm worried about this making the build red (well, more red) for everyone.

dellis23 avatar Oct 10 '22 15:10 dellis23

just use the "Oneshot build" and select the pypi-release branch. It will do a full build for you and do a release.

powderluv avatar Oct 10 '22 15:10 powderluv

For the large file issue -- we can comment out auditwheel for now and see if you can upload it. Auditwheel copies the dylibs from pytorch so the file size increases.

Long term we have to shed the weak-link into Pytorch and build statically just what we need (not all Aten ops)

powderluv avatar Oct 14 '22 20:10 powderluv

Dropping the auditwheel gives this:

ERROR    HTTPError: 400 Bad Request from https://upload.pypi.org/legacy/        
         Binary wheel 'torch_mlir-20221017.60-cp310-cp310-linux_x86_64.whl' has 
         an unsupported platform tag 'linux_x86_64'.

Unfortunately, the suggested fix seems to be to use auditwheel.

dellis23 avatar Oct 17 '22 20:10 dellis23

It looks like there's a way to request an exemption on the file size limit: https://pypi.org/help/#file-size-limit

I can file a request. We need to include the total size of the project (I assume 200MB, based on the output I had from before?) and a reason for the additional size. Do you have a compelling statement I could use for the latter part? I don't know enough about everything going into the build to be able to say anything other than "the project is big."

dellis23 avatar Oct 17 '22 20:10 dellis23

How about :

torch-mlir depends on C++ libraries that are included as part of its wheel to be able to provide maximum compatibility flexibility for the end users.

powderluv avatar Oct 17 '22 21:10 powderluv

Filed https://github.com/pypa/pypi-support/issues/2307

Edit: also filed https://github.com/pypa/pypi-support/issues/2374

dellis23 avatar Oct 18 '22 13:10 dellis23

Thank you

powderluv avatar Dec 07 '22 15:12 powderluv