Distributed.jl

Distributed.jl copied to clipboard

Here's the code to reproduce (Julia 1.1.0):

using Distributed
addprocs(2)
f = (;x)->2x
pmap(1:2) do i
    f(x=i)
end

This will give an error like:

ERROR: On worker 2:
UndefVarError: ##7#8 not defined

As far as I can tell reading the docs, this should work. Also note that it does work if the function has no keyword arguments.

f = x->2x
pmap(1:2) do i
    f(i)
end

2-element Array{Int64,1}:
 2
 4

Any chance this could get added as a 1.3 milestone? It appears to still be present on current master.

If all bugs got milestoned we could never do a release. This doesn't seem to be a regression so it is unlikely that we will hold the 1.3 release for it to be fixed (which is what the milestone means).

Fair enough, I just figured this was parallel-y enough that it might make sense for 1.3 which emphasizes that. FWIW, if there's an easy path to fixing this that involves someone pointing a non-expert like me in the general right direction, I could give it a go, although maybe its easiest to just get the right person's attention for a bit.

For posterity / anyone else running into this, here's two workarounds, either bypass the auto-global-variable distribution by making it be closed over:

let f = (;x)->2x
    pmap(1:2) do i
        f(x=i)
    end
end

or, well, I don't know why this one works:

get_f() = (;x)->2x
f = get_f()
pmap(1:2) do i
    f(x=i)
end

It would be really nice if this were fixed, or at least documented. I lost quite a few hours this week discovering and trying to circumvent it.