DistributedArrays.jl
DistributedArrays.jl copied to clipboard
JuliaParallel
Distributed Arrays in Julia
One of the current issues with DArray is that each operation is immediately synchronizing. Requiring a distributed operation to finish before we can carry on with scheduling new operations. This...
In #175 I prototyped supporting CuArray with DArray. That works needs to mature and be properly tested, as well as take advantage of local IPC on the same machine.
Right now `DArray` is both the notion of where and the data. I propose to factor out the where into it's own Partitons type, so that we can have a...
Dear All, I'm considering the following MWE, which consists of creating a matrix and vector of a `struct` an doing a calculation on them. I import the relevant mathematical operators...
``` julia> using Distributed julia> addprocs(2); julia> @everywhere using DistributedArrays julia> a = fill(1000,10); julia> da = distribute(a); julia> @time map(x->rand(x,x)^2, a); 0.903241 seconds (63.61 k allocations: 155.698 MiB, 29.39%...
I believe this used to work, but now gives ``` julia> distribute(1:100) MethodError: no method matching UnitRange{Int64}(::Array{Int64,1}) Closest candidates are: UnitRange{Int64}(::Any, !Matched::Any) where T
``` A = drand(64, 64) A[1:10] .= 1 ``` Will create a `View{ReshapedArray{DArray{Array}}}` this multiple levels of nesting is quite nasty and annoying. We either need to deal with ReshapedArrays...
The function name `scatter` in DistributedArrays conflicts with the `scatter` function in, say, 80% of all julia's plotting packages (for making scatter plots). Would it be possible to rename to...
Is it feasible that DArray could support distributed arrays in which pieces are missing? For instance, say I have the following: ``` X X X X X X X X...
I build a DArray as follows: ``` @everywhere using DistributedArrays r1 = @spawnat 2 zeros(4,4) r2 = @spawnat 2 zeros(4,4) r3 = @spawnat 2 rand(4,4) r4 = @spawnat 3 rand(4,4)...