ridgeplot icon indicating copy to clipboard operation
ridgeplot copied to clipboard

Fix `test_regressions` tests failing in GitHub Actions

Open tpvasconcelos opened this issue 1 year ago • 1 comments

I had to mark the test_regressions tests with XFAIL when running the checks in CI because they are currently failing when being run in GitHub Actions. See here and here.

This is probably due to small differences in output between environments (local vs. remote macOS, Windows, and Linux).

The test, for reference: https://github.com/tpvasconcelos/ridgeplot/blob/34498e54b2381ed97ac09f182df5a40f8e4a4943/tests/e2e/test_examples.py#L30-L37

Regardless of the solution, one nice feature to implement here is the ability to upload the plot artefacts that resulted in the failed tests and expose these artefacts to the relevant PR. This discussion might be useful for this:

  • https://github.com/actions/upload-artifact/issues/50 other references on this topic:
  • https://github.com/orgs/community/discussions/51403
  • https://github.com/djthornton1212/djthornton1212/blob/main/.github/workflows/pr-comment-artifact-url.yml
  • https://stackoverflow.com/questions/60789862/url-of-the-last-artifact-of-a-github-action-build

tpvasconcelos avatar Jan 08 '24 22:01 tpvasconcelos

For some reason, the tests/e2e/test_examples.py::test_examples_width_height_set[lincoln_weather-load_lincoln_weather] hangs forever and never finishes.

Some examples here: https://github.com/tpvasconcelos/ridgeplot/actions/runs/7481625128/job/20363565240

This ran succeeded but it took over 14minutes to run the e2e test suite: https://github.com/tpvasconcelos/ridgeplot/actions/runs/7481625128/job/20363565240

For this reason, I have also changed the xfail mark to a skipif (https://github.com/tpvasconcelos/ridgeplot/pull/163/commits/8faa9df224c1b35c95385e5144936736d016fb25)

tpvasconcelos avatar Jan 10 '24 22:01 tpvasconcelos