liboqs
liboqs copied to clipboard
open-quantum-safe
Expand weekly test runs to platforms other than x86_64 / Linux
After https://github.com/open-quantum-safe/liboqs/pull/1560 lands, the full KAT tests will be run on a weekly basis. Our weekly jobs currently (the KAT tests and the constant-time tests) only run on x86_64 / Linux at the moment. It would be nice to also run the tests on aarch64, Windows, MacOS, etc.
I'm not familar with circleci. So I think only x86_64 is used for the current github workflows? They do provide runners for some of these options.
Some platforms I don't see include
- ubuntu on arm (aarch64)
- Z (s390x)
Either would require external runners - there's tools to set these up, but need to find someone to provide a suitable pool?
So I think only x86_64 is used for the current github workflows?
Correct. And that (desire to test on other HW) is the primary reason why we still have to support CircleCI.
need to find someone to provide a suitable pool?
Correct. That was one of the prime motivations for OQS to join LF (presumably easier access to more platforms for CI). Right now, those IBM platforms are tested via Travis. And we'd love to streamline our CI to one system (incl. getting rid of CCI), so any contribution very welcome.
Some platforms I don't see include
- ubuntu on arm (aarch64)
- Z (s390x)
And we'd love to streamline our CI to one system
Moving everything to one CI would be great. @planetf1, as Michael says we currently have basic s390x and ppc64le CI with travis, which is sometimes a bit flaky (I have a ticket open with Travis about the intermittent errors). There is work to allow self-hosted runners on linux-s390x that might be worth to follow: https://github.com/actions/runner/issues/2263.
.. or https://github.com/marketplace/actions/run-on-architecture which seems to support both s390x, ppc64le (and more).
When the pqca was forming we did get a post from a github employee suggesting we might be able to get some extra help with github
- There is an unofficial version of github action runner as mentioned above, which seems current, and now supports (experimental...!) s390x. There is hope this could get integrated by github. maybe we can point them to this ? (@bhess presumably we could find some s390x to actually run on?)
- ubuntu arm64 runners went into private beta in Jan 24 - perhaps we could get access to this
What else are we missing? I will try and followup on that offer with gh/lf ...
I have talked to GitHub about getting into the Arm private beta, but it was pending some administrative work to get our account upgraded to Enterprise. Pinging @ryjones for any updates on that front.
- There is an unofficial version of github action runner as mentioned above, which seems current, and now supports (experimental...!) s390x. There is hope this could get integrated by github. maybe we can point them to this ? (@bhess presumably we could find some s390x to actually run on?)
This would be great @planetf1 and IMO the way to go when consolidating to Github CI. I also tried an experimental build using run-on-architecture and s390x.. it works but it's terribly slow because it's emulated. So not really an option, also in light of the project's goal for responsible CI use.
@SWilson4 no follow up, yet. I'm as annoyed as you are :) @bhess if your s390x vms/etc are available to travis, it should be easy enough to hook them into github.
I have talked to GitHub about getting into the Arm private beta, but it was pending some administrative work to get our account upgraded to Enterprise. Pinging @ryjones for any updates on that front.
GitHub folks contacted me to say that PQCA is now on GitHub Enterprise, so I'm hopeful that Arm runners are coming soon.
@SWilson4 working on it :) Right now we have large runners, but not arm (yet)