pytest-xdist
pytest-xdist copied to clipboard
pytest-dev
Lock not take effect while being used in test run parallelly.
Hi, I was trying to run some test cases that needs to be run with resource exclusive while others in parallel execution. The way I prevent each worker process to access the resource is putting a lock by using python's Lock object, like:
lock.acquire() #do something exclusive lock.release()
the lock is created either in a session wide fixture or global variable in the .py file, but neither take effect. I was wonder if lock itself work with x-dist?
then what about multiprocessing.Lock or filelock? I'm wondering if there is a way to force some of x-dist worker processes run one by one?
The docs has examples with file locks, Please look them up (it's a pain on my smartphone )
@qwesadasuidhfaiusdhfi
install filelock using pip install, now you can set up a file lock to keep single process doing the job.
from filelock import FileLock with FileLock('filename.lock'): pass # do some exclusive things
any other threads can not start to work until current one finishes the job and exits with block.
Hi , FileLock works fine for me , is there a way that pytest-xdist works with multiprocess.Lock ?
Multiprocess locks don't work unless multiprocessing is actually configured for it
what do you mean by configured for it ? is there a way to pass the multiprocessing lock to the workers created by pytest-xdist ? or that not supported
multiprocessing locks will only work if you spawn the subprocesses using multiprocessing (with Pool, Process, etc), it will not work with processes spawned by other means -- pytest-xdist uses execnet to spawn the processes.
