Amit Murthy

The leak maybe in libuv like https://github.com/JuliaLang/julia/issues/13529 probably is due to a libuv issue.

Defining this should suffice, right? ``` distribute(d::DArray) = DArray(I->convert(Array, d[I...]), d) ```

I modified `verified_destination_serializer` in a local branch, but I ended up needing changes in `distribute` too. I feel it is cleaner and better to add methods specific to `DArray`.

I think this is related to matrix multiplication using multi threaded BLAS on the master but single threaded BLAS on the workers. ``` julia> using Distributed julia> addprocs(4) 4-element Array{Int64,1}:...

Setting num BLAS threads to 1 on the master shows distributed map scaling reasonably well for larger workloads. ``` julia> using Distributed julia> addprocs(4); julia> @everywhere using DistributedArrays julia> using...

Timing `map` on DArray from within a function is vastly faster than outside. Though this is not the behavior for a local `map`. ``` julia> using Distributed julia> addprocs(4); julia>...

Should the split packages be with individual contributors or under `JuliaParallel` ? The maintainers of the separate cluster managers ought to be users of the specific manager.

@andreasnoack

I am not familiar with HDF5 and its capabilities (specifically with distributing a single variable write across node), however, in a generic sense and using `spmd` pseudo code should be...

Yes the bug is that `similar` is not having the same distribution. A workaround is to use the default distribution or explicitly create the second darray and `copy!` into it.