pyquarkchain icon indicating copy to clipboard operation
pyquarkchain copied to clipboard

pytest fails with "too many open files"

Open qcdll opened this issue 5 years ago • 1 comments

without setting ulimit -Sn 10000 pytest will fail with lots of "too many open files" error looks originated from test_cluster.py, need to investigate

qcdll avatar Mar 15 '19 18:03 qcdll

add the following snippet to test_cluster.py, we see what fd are being leaked: a few samples

pypy3   63155  dll  159     PIPE 0x60bc88493f1ee7d3    16384          ->0x60bc88493f1f0093
pypy3   63155  dll  160     PIPE 0x60bc88493f1f0093    16384          ->0x60bc88493f1ee7d3
pypy3   63155  dll  161r  PSXSEM                         0t0          /mp-3kf8shak
pypy3   63155  dll  162r  PSXSEM                         0t0          /mp-7w1xhlq3
pypy3   63155  dll  163r  PSXSEM                         0t0          /mp-4p58kng9

I suspect there are unclean shutdown of slave code that did not properly close fds PSXSEM means POSIX semaphore file

qcdll avatar Apr 18 '19 23:04 qcdll