pandas icon indicating copy to clipboard operation
pandas copied to clipboard

BLD: Build wheels using cibuildwheel

Open lithomas1 opened this issue 3 years ago • 11 comments

  • [ ] closes #44027 (Replace xxxx with the Github issue number)
  • [ ] Tests added and passed if fixing a bug or adding a new feature
  • [ ] All code checks passed.
  • [ ] Added type annotations to new arguments/methods/functions.
  • [ ] Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

We can't switch to this instantly since there's still a lot of work to be done, but this is a good start(the wheels build). This PR can be merged anytime, though, and separate PRs can be put up to address issues.

The upload script probably won't work(it should, though, it is copy pasted from numpy's), but the only way to check is to merge this.

Current issues:

  • [ ] Align flags when building wheels ( the wheel sizes are way off, I need to turn on stripping for sure)
  • [ ] Figure out licensing for MSVC redistributables (I don't think we were doing this correctly before)
  • [x] An admin needs to add the anaconda keys.
  • [ ] aarch64 wheel builds
  • [ ] adjust the flaker rules

lithomas1 avatar Aug 27 '22 19:08 lithomas1

Looks pretty good to start. Could you show a commit where the workflow ran (and maybe save the wheel as an artifact)?

mroeschke avatar Sep 09 '22 18:09 mroeschke

If you scroll down on https://github.com/pandas-dev/pandas/actions/runs/3029167054, you can find the artifacts.

Note that the manylinux wheels are much larger, since I haven't passed the flags to strip wheels yet. I'll try to match flags in a follow up PR. (The macosx wheels are normal size, even though they seem bigger since both the x86_64 and arm64 wheels are included in the artifact)

lithomas1 avatar Sep 13 '22 11:09 lithomas1

The last remaining blocker I think is for someone with access to the Anaconda org to add the keys to this repo.

lithomas1 avatar Sep 14 '22 22:09 lithomas1

@TomAugspurger Sorry for the ping, but do you know how to get the tokens for the staging and nightly anaconda uploads? I think you were the one to add the tokens originally to the pandas-wheels repo.

lithomas1 avatar Sep 20 '22 14:09 lithomas1

No worries.

I think anyone in the scipy-wheels-nightly group on anaconda.org (https://anaconda.org/scipy-wheels-nightly) can generate a token for uploading or view the existing tokens. Do you have an account on anaconda.org @lithomas1?

TomAugspurger avatar Sep 20 '22 15:09 TomAugspurger

Oh, or I can just set the environment variable in this repo. Let me do that quick.

TomAugspurger avatar Sep 20 '22 15:09 TomAugspurger

@lithomas1 there's now a PANDAS_NIGHTLY_UPLOAD_TOKEN secret in GitHub that should have permission to upload to that group in anaconda.org.

TomAugspurger avatar Sep 20 '22 15:09 TomAugspurger

@lithomas1 there's now a PANDAS_NIGHTLY_UPLOAD_TOKEN secret in GitHub that should have permission to upload to that group in anaconda.org.

Thanks, can you also add the token for multibuild-wheels-staging under PANDAS_STAGING_UPLOAD_TOKEN?

lithomas1 avatar Sep 20 '22 16:09 lithomas1

Done!

TomAugspurger avatar Sep 20 '22 16:09 TomAugspurger

@WillAyd Can you re-review again when you have time? I think your review is blocking the merge.

(No rush though, this can go in anytime before 1.6/2.0)

lithomas1 avatar Sep 20 '22 20:09 lithomas1

@lithomas1 sorry about that. Let's see how this goes - thanks for the ongoing improvements

WillAyd avatar Sep 20 '22 21:09 WillAyd

Going to put this in now given the approvals. Let's see how the nightlies work out this night.

lithomas1 avatar Sep 21 '22 18:09 lithomas1