JLD.jl
JLD.jl copied to clipboard
Loading SharedArray's doesn't redistribute the array across workers
trafficstars
I just noticed that when I save a SharedArray, the type information is appropriately recreated on load. However, when loaded, the data exists in the primary thread, but is not shared across all the other active workers, as in normally done when creating a SharedArray.
Saving the data:
addprocs(1)
using JLD
data = SharedArray{Float64,1}(rand(100))
# The data exists on the worker thread
fetch(@spawn data)
jldopen("test.jld","w") do f
write(f,"test",data)
end
Loading the data (make sure it's a different process):
addprocs(1)
using JLD
data = jldopen("test.jld") do f
read(f,"test")
end
fetch(@spawn data)
The final line there throws this error:
ERROR: LoadError: On worker 2:
Local instance of remote reference not found
Adding data = SharedArray{Float64,1}(data) doesn't fix it, because it's already a SharedArray. Converting it to an Array and then back does fix it, using data = SharedArray{Float64,1}(copy(data)).
Would the best solution be to add a simple conditional that catches SharedArray's, and then reinitializes them through that workaround?