JuliaIO
thread safety?
could someone please elaborate on @timholy 's concern about thread safety here: https://github.com/timholy/HDF5.jl/pull/132
i ask, because i've been having a problem writing .mat files from julia inside a pmap. julia's added procs are processes, not threads, correct? and if each process writes to a separate .mat file, shouldn't all be okay? yet doing so, what was eventually getting stored to disk was on occasion not what i wrote to it. strings were getting truncated or pre-pended to. numbers were changing values. all in a non-deterministic way. only single-threading matwrite by passing the data back to the main thread solved the problem.
Yes, Julia uses processes for pmap. If you see a problem while accessing different files from different processes, then this can also be a problem with the file server -- I've seen systems where even reading (!) 100 files in parallel would return wrong results. That's obviously a system problem, I assume the NFS server was at fault.
From your description it sounds as if HDF5 is not at fault.
