catalyst icon indicating copy to clipboard operation
catalyst copied to clipboard

Running async tests with `pytest -n auto`

Open erick-xanadu opened this issue 1 year ago • 0 comments

While looking into exception handling I found that async tests (even without triggering exceptions and without the pass for exception handling) would not consistently succeed when load was increased. I copy pasted the async tests (without exceptions) and parametrized them to run 16 times each. I then used variations of this test to determine when they failed:

#!/usr/bin/env bash

set -euo pipefail

for i in {1..100} ; do
  pytest ./frontend/test/pytest/test_async.py -n auto
  echo ${i}
done

When running these async tests, it never succeed with -n auto but a test running 300 times without -n auto succeeded.

I will attempt to minimize bug and contact the LLVM community to see if we can narrow things down.

erick-xanadu avatar Jan 15 '24 15:01 erick-xanadu