pydata-sphinx-theme icon indicating copy to clipboard operation
pydata-sphinx-theme copied to clipboard

MAINT: proposed CI improvements

Open drammock opened this issue 1 year ago • 1 comments

Our a11y test currently runs as part of the pytest matrix, but only on one of the matrix jobs (py3.11 / ubuntu-latest). It's marked with continue-on-error so that its failure doesn't prevent subsequent steps from running.

It turns out that continue-on-error has the unfortunate side-effect of marking the job as successful (!) so we'll need to click through to the CI log to actually see whether there were a11y test failures. Apparently an alternative is to mark any subsequent steps in the job as if: always() --- this lets subsequent steps run while still marking the failed step with a red X. However:

  • the a11y test is currently the last step in its job anyway so we probably don't need either continue-on-error or if: always()
  • the a11y test should probably be its own job rather than a step at the end of the main pytest job.

Originally posted by @drammock in https://github.com/pydata/pydata-sphinx-theme/issues/1320#issuecomment-1542471166

drammock avatar May 10 '23 16:05 drammock