ibis icon indicating copy to clipboard operation
ibis copied to clipboard

ci: improve when we run the full test suite on CI

Open gforsyth opened this issue 1 year ago • 1 comments

Current status

We have a large test suite and don't want to run the entire thing for unrelated changes. Currently, there are two ways that CI gets run:

  1. Run everything
  2. If all the changes in a PR (or push) are within the docs/ folder (or are one of a few config files that lives at the root of the repo), then only run a subset of the test suite (mostly the docs-related bits).

This is a good start, but I think we can probably reduce the testing burden more with a few rules that should give us reasonably good coverage (I think).

New rules

  • Keep the current "docs-only" rule, but also add in #8231 to allow for adding words to the codespell dictionary without triggering a full CI run
  • For changes in ibis/expr/: run everything
  • For changes in ibis/backends/base: run everything
  • For changes in ibis/backends/<backend name>: only run the tests for that backend
    • An exception to this rule is Risingwave, which is mostly the postgres backend and so changes to postgres should also trigger risingwave tests

Then, as a fallback, we add a label akin to our docs-preview label that can trigger a full backend test suite run if the reviewer thinks that something might have a larger blast radius than accounted for by these rules.

gforsyth avatar Feb 05 '24 20:02 gforsyth

I just found out about path-filters GHA thanks to this blog https://www.flightcontrol.dev/blog/reduced-github-actions-bill-by-63.

I think we could use this to filter by backend folders.

ncclementi avatar Feb 20 '24 16:02 ncclementi