catalyst
catalyst copied to clipboard
Running async tests with `pytest -n auto`
While looking into exception handling I found that async tests (even without triggering exceptions and without the pass for exception handling) would not consistently succeed when load was increased. I copy pasted the async tests (without exceptions) and parametrized them to run 16 times each. I then used variations of this test to determine when they failed:
#!/usr/bin/env bash
set -euo pipefail
for i in {1..100} ; do
pytest ./frontend/test/pytest/test_async.py -n auto
echo ${i}
done
When running these async tests, it never succeed with -n auto but a test running 300 times without -n auto succeeded.
I will attempt to minimize bug and contact the LLVM community to see if we can narrow things down.