psautohint icon indicating copy to clipboard operation
psautohint copied to clipboard

pytest-randomly causes occasional CI failures

Open josh-hadley opened this issue 2 years ago • 1 comments

While attempting to sync dependencies between dev-requirements.txt and setup.py, I re-introduced pytest-randomly which was listed in setup.py but not in dev-requirements.txt. When running the test suite in CI, some test runs failed on the test_hashmap_old_version and test_hashmap_no_version tests (but were occasionally successful upon running again). I concluded that the random test execution order introduced by pytest-randomly was the cause, and removed it. This should be investigated as it suggests that there is some precondition that is getting set when the tests are run in the defined order but which is not always met when run randomly.

josh-hadley avatar Jul 24 '22 03:07 josh-hadley