alphabase icon indicating copy to clipboard operation
alphabase copied to clipboard

Race condition when reading speclib from two instances

Open mschwoer opened this issue 8 months ago • 7 comments

Describe the bug When two alphaDIA instance access the same speclib file (for reading) in the same instance of time, alphabase throws an error (see below).

Expected behavior No error, as the file is just opened for reading in this cases I think the problem is that files are opened for 'a' here (hdf.py)

class HDF_File(HDF_Group):
    def __init__():
...
        mode = "w" if delete_existing else "a"
        with h5py.File(file_name, mode):  # , swmr=True):
            pass

Logs

[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO - 0:00:00.022445 INFO: Running DynamicLoader
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO - 0:00:00.027390 INFO: Loading .hdf library from /fs/hela_hybrid.small.hdf
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO - 0:00:00.031234 INFO: Traceback (most recent call last):
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -   File "/fs/home/xx/conda-envs/alphadia-1.6.2/lib/python3.11/site-packages/alphadia/cli.py", line 333, in run
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -     plan = Plan(
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -            ^^^^^
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -   File "/fs/home/kraken/conda-envs/alphadia-1.6.2/lib/python3.11/site-packages/alphadia/planning.py", line 126, in __init__
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -     self.load_library()
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -   File "/fs/home/kraken/conda-envs/alphadia-1.6.2/lib/python3.11/site-packages/alphadia/planning.py", line 205, in load_library
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -     spectral_library = dynamic_loader(self.library_path)
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -   File "/fs/home/kraken/conda-envs/alphadia-1.6.2/lib/python3.11/site-packages/alphadia/libtransform.py", line 40, in __call__
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -     return self.forward(*args)
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -            ^^^^^^^^^^^^^^^^^^^
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -   File "/fs/home/kraken/conda-envs/alphadia-1.6.2/lib/python3.11/site-packages/alphadia/libtransform.py", line 121, in forward
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -     library.load_hdf(input_path, load_mod_seq=True)
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -   File "/fs/home/kraken/conda-envs/alphadia-1.6.2/lib/python3.11/site-packages/alphabase/spectral_library/base.py", line 681, in load_hdf
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -     _hdf = HDF_File(
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -            ^^^^^^^^^
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -   File "/fs/home/kraken/conda-envs/alphadia-1.6.2/lib/python3.11/site-packages/alphabase/io/hdf.py", line 533, in __init__
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -     with h5py.File(file_name, mode):#, swmr=True):
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -          ^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -   File "/fs/home/kraken/conda-envs/alphadia-1.6.2/lib/python3.11/site-packages/h5py/_hl/files.py", line 562, in __init__
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -     fid = make_fid(name, mode, userblock_size, fapl, fcpl, swmr=swmr)
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -   File "/fs/home/kraken/conda-envs/alphadia-1.6.2/lib/python3.11/site-packages/h5py/_hl/files.py", line 247, in make_fid
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -     fid = h5f.open(name, h5f.ACC_RDWR, fapl=fapl)
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -   File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -   File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO -   File "h5py/h5f.pyx", line 102, in h5py.h5f.open
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO - BlockingIOError: [Errno 11] Unable to synchronously open file (unable to lock file, errno = 11, error message = 'Resource temporarily unavailable')
[2024-06-12, 15:19:11 UTC] {ssh.py:526} INFO - 

mschwoer avatar Jun 12 '24 17:06 mschwoer