DistributedArrays.jl icon indicating copy to clipboard operation
DistributedArrays.jl copied to clipboard

Multiple chunks on one process

Open r-barnes opened this issue 7 years ago • 1 comments

I build a DArray as follows:

@everywhere using DistributedArrays
r1  = @spawnat 2 zeros(4,4) 
r2  = @spawnat 2 zeros(4,4) 
r3  = @spawnat 2 rand(4,4)
r4  = @spawnat 3 rand(4,4)
ras = [r1 r2; r3 r4]
D   = DArray(ras)

My expectation is that the output should have two 4x4 blocks of zeros and two 4x4 blocks of random numbers. Instead, I get three blocks of zeros and one block of random numbers:

8×8 DArray{Float64,2,Array{Float64,2}}:
 0.0  0.0  0.0  0.0  0.0         0.0       0.0       0.0     
 0.0  0.0  0.0  0.0  0.0         0.0       0.0       0.0     
 0.0  0.0  0.0  0.0  0.0         0.0       0.0       0.0     
 0.0  0.0  0.0  0.0  0.0         0.0       0.0       0.0     
 0.0  0.0  0.0  0.0  0.525043    0.321681  0.489682  0.586815
 0.0  0.0  0.0  0.0  0.569794    0.780382  0.542156  0.215128
 0.0  0.0  0.0  0.0  0.00308504  0.912877  0.179453  0.568009
 0.0  0.0  0.0  0.0  0.410886    0.333188  0.743346  0.969894

r-barnes avatar Oct 01 '18 19:10 r-barnes

Jup that is a bug in the constructor: https://github.com/JuliaParallel/DistributedArrays.jl/blob/e6915d28b59f564043597985bd5af1896f3e380b/src/darray.jl#L110

localpartindex(pids) assumes that there is only one chunk on a process and will return the first one.

vchuravy avatar Oct 01 '18 20:10 vchuravy