tornado icon indicating copy to clipboard operation
tornado copied to clipboard

ci: add riscv64 manylinux/musllinux wheels

Open justeph opened this issue 1 month ago • 3 comments

Proposed new feature or change:

I'm opening this issue to start the discussion on what it would take to get the tornado project to build and distribute riscv64 manylinux images.

Why now?

Until recently, it wasn't really possible for Python projects to build binary riscv64 wheels and to distribute those wheels via PyPI. Manylinux and cibuildwheel did not support riscv64 and PyPI did not allow riscv64 wheels to be uploaded. This all changed during the summer of 2025, when cibuildwheel (3.12), manylinux and warehouse all gained riscv64 support. Encouragingly, some projects (lxml, uv, maturin, and critically ninja) have already started uploading riscv64 wheels to PyPI.

Does tornado work on riscv64?

Yes. RISE has been building and distributing tornado manylinux wheels for riscv64 for over a year (since version 6.4.1). Tested builds are available for riscv64 here.

What needs to be done?

If I am correct, there is no available native riscv-64 runner in GitHub. So build and test will involve emulation though QEMU. I already did some tests on my end and we need to:

  • skip test_unquote_large and test_request_timeout
  • increase ASYNC_TEST_TIMEOUT to 30 seconds because of emulation

Only a few additions to the github action file is needed to build and test a riscv64 wheel. We also need a fairly recent version of cibuildwheel, to fix python 3.14 issues on riscv64

Note: riscv64 builds are slower due to emulation and currently take about 30 minutes.

I have a PR ready for review.

How can RISE help?

RISE is aware of the additional burden of supporting a new architecture, as mentioned https://github.com/scientific-python/summit-2025-nov/issues/4. The RISE project should be able to assist by providing engineering resources, to debug riscv64 specific issues, and potentially native riscv64 runners, if required.

justeph avatar Nov 27 '25 13:11 justeph

It would be annoying to slow down the build after we just removed qemu for ARM (which got build times down to 15-18 minutes), but other than that if it's just a matter of building with qemu I don't really have strong objections (it wasn't obvious to me that this was available for platforms other than arm). That pattern was reliable on our ARM builds and we never had any issues with it (it helps that our C extension is as trivial as it gets).

bdarnell avatar Dec 03 '25 19:12 bdarnell

Hi @bdarnell , thank you for feedback. As mentioned above RISE is working on providing native RISC-V builders for open-source projects but they're not ready yet. Would you be interested in such builder?

justeph avatar Dec 04 '25 16:12 justeph

Our build is small enough that emulation doesn't cost too much time (and we only do it in the release build pipeline), so I'd prioritize simplicity and reliability over speed. I'd rather use emulation for all architectures that don't have github-hosted runners (we've had requests for ppc64le in #3449) than work with third parties for native builders.

bdarnell avatar Dec 04 '25 22:12 bdarnell

Hi @bdarnell, is there any chance to merge riscv64 support? Are you waiting for more input from me? Can I help on something else?

justeph avatar Dec 15 '25 14:12 justeph

Yes, I'm planning to merge this and also add emulated builds for other platforms. But it's low priority at the moment since it wouldn't go out until our next feature release which is at least months away.

bdarnell avatar Dec 15 '25 18:12 bdarnell