cloudflare
Generate python import tests in bazel
Prerequisite: #1970
Includes: #2108
This PR adds dozens of wd-tests generated by Bazel, based off a list generated by pyodide-build-scripts from the pyodide meta.yaml files and added onto pyodide_bucket.bzl.
Each test simply has the tested package listed as a requirement and a worker.py script that just imports a bunch of stuff from that package. I expect it will take a while to get all these tests passing but we do need them to pass eventually.
Code LGTM, I think auto-generating tests here will be really useful in improving python test coverage. I hate to always bring up resource constraints but the ~50 tests being added all take between 20 - 50 seconds each – with the GitHub Actions runners being limited to 4 vCPUs each, that will slow down CI on every run making a code change. Is there any potential to optimize python-based tests?
Could we gate the tests to only run on changes inside the pyodide folders?
Could we gate the tests to only run on changes inside the pyodide folders?
Good idea, anyone know how to do that? 🫣
Maybe Bazel's cache could be useful if someone could point me to how to configure it 😮💨
Is it possible to run the tests off a baseline snapshot as well? @hoodmane @dom96
Is it possible to run the tests off a baseline snapshot as well? @hoodmane @dom96
It's not something that we have set up, but it could be done. We have lots of tests using different kinds of snapshots in the EW repo, might be easier to do there.
@fhanau This PR is ready for review again after changes -- the tests are now configured to only run in the (linux) configuration, which was chosen because it was the fastest-running test pipeline. The idea is that the ~5 extra minutes spent on that pipeline would end up being spent on the other pipelines anyway.
Will require similar CI changes upstream.