JuliaLang
Surprising and undocumented `import` handling in `Distributed.@everywhere`
Consider the following code:
using Distributed
addprocs(1)
@everywhere begin
# assume MyModule is defined in the path given
push!(LOAD_PATH, "/path/to/my/module")
import MyModule
end
This fails with a LoadError, because @everywhere cherry-picks the imports for each everywhere block and runs them first regardless of the order of statements inside the block. Instead, one has to do this:
# (...)
@everywhere push!(LOAD_PATH, "/path/to/my/module")
@everywhere import MyModule
This behaviour thoroughly breaks my internal parser, because these two blocks look equivalent (modulo an unnecessary synchronization point in the middle). Worse, the behaviour is not documented, and in fact contradicts the documentation.
Unfortunately, no rationale is given in the code as to why this was chosen. However, I think at the very least, one should amend the documentation with a corresponding note and add a rationale for this in the code.
One could maybe also issue a warning for @everywhere blocks which do not have import statements as their first statements. This would help poor newbies like me :)
Julia version: 1.2.0 (also present in master)
Thank you!! I was getting very strange errors LoadError and UndefVarError in my worker processes, as if modules I loaded in
@everywhere begin
import Random
using Printf
using GraphPlot
using JuMP
import CSV
using Profile
<other non-import-related code here>
end
were not actually loaded at all. Putting @everywhere at the beginning of just the import and using lines fixed it. All my other code (functions, variables, etc) was fine inside a single @everywhere begin .... end block. Thanks for pointing this out, hope others find this useful.
