Modules unreachable from Main in julia 0.7

[carlobaldassi]

I was testing #215 and noticed a difference between 0.6 and 0.7 which doesn't show up in the test files, and affects loading from within modules (and affects JLD2 too, from what I can tell.)

It's about modules being reachable from Main even though they were loaded (with using or import) from within a different module. This used to be the case in 0.6, so that this line was working fine. It's no longer like this in 0.7. Here's a minimal example which can be run from the REPL:

julia> module Tst
           using ArgParse
           T = ArgParseSettings
           T2 = Core.eval(Main, :(ArgParse.ArgParseSettings))
           @show T, T2
       end

The Core.eval line works fine in 0.6 but fails in 0.7, because ArgParse is not reachable from Main. Is there a way to find the module anyway in 0.7? I assume not, since there may be different modules with the same name loaded from within different modules, right?

So now something like this doesn't work any more:

module Tst
    using FileIO, MyModule
    fn = "somefile.jld"
    if isfile(fn)
        x = load(fn, "x") # fails to recognize type MyModule.MyObject
    else
        x = MyObject()
        save(fn, "x")
    end
    # use x
end

For the time being, I'm solving this issue by doing things like Core.eval(Main, :(import MyModule)) before calling load on a file which was saved with some MyModule object in it. this is clearly not great. Alternatively, we could set up things to pass an optional module name to load, which defaults to Main but would actually be given as @__MODULE__ most of the times. Any other suggestions? Can load be made aware of the module it was called from?

[JeffBezanson]

Fixed by #227 I think?

[carlobaldassi]

It's not fixed unfortunately. Here's a miminal example to reproduce, that mimics my use case. Save this into a file "jtst.jl":

module JTst

using FileIO, ArgParse

function getx()
    fn = "somefile.jld"
    if isfile(fn)
        # Core.eval(Main, :(import ArgParse)) # this "fixes" the issue
        x = load(fn, "x") # fails to recognize type ArgParse.ArgParseError
    else
        x = ArgParseError("")
        save(fn, "x", x)
    end
    return x
end

end

Then run it twice:

julia-1.0> include("jtst.jl")
Main.JTst

julia-1.0> JTst.getx()
ArgParse.ArgParseError("")

julia-1.0> JTst.getx()
┌ Warning: type ArgParse.ArgParseError not present in workspace; reconstructing
└ @ JLD ~/.julia/dev/JLD/src/jld_types.jl:707
getfield(JLD, Symbol("##ArgParse.ArgParseError#360"))("")

[guilhermebodin]

I am also having this with Dates from stdlib, I am running on Julia 1.0.3

[guilhermebodin]

I created a JLD doing

using Dates, JLD
date = DateTime(2019)
save("dates.jld", "date", date)

then killed julia and defined a module Tst

module Tst

    using Dates, JLD

    export load_dates

    function load_dates(file)
        JLD.load(file)
    end

end

Tst.load_dates("dates.jld")

and this gives the following

julia> Tst.load_dates("dates.jld")
┌ Warning: type Dates.DateTime not present in workspace; reconstructing
└ @ JLD ~/.julia/packages/JLD/1BoSz/src/jld_types.jl:703
┌ Warning: type Dates.UTInstant{Dates.Millisecond} not present in workspace; reconstructing
└ @ JLD ~/.julia/packages/JLD/1BoSz/src/jld_types.jl:703
┌ Warning: type Dates.Millisecond not present in workspace; reconstructing
└ @ JLD ~/.julia/packages/JLD/1BoSz/src/jld_types.jl:703
Dict{String,Any} with 1 entry:
  "date" => ##Dates.DateTime#361(##Dates.UTInstant{Dates.Millisecond}#362(##Dates.Millisecond#363(63681984000000)))

The only way to solve this is by adding Dates before defining the module Tst but this is not ideal for many applications

using Dates

module Tst

    using Dates, JLD

    export load_dates

    function load_dates(file)
        JLD.load(file)
    end

end

Tst.load_dates("dates.jld")

[jonbarkerlondon]

I'm having this problem also when using PackageCompiler to build an .EXE and then load an existing JLD - the module path for Dates changes, and therefore JLD can't load the type and reconstructs it.

[jonbarkerlondon]

@guilhermebodin if you need a quick hackfix for this, then I was able to get everything working perfectly.

PackageCompiler produces all my structs etc, and imports modules like 'Dates' under Main.Mod instead of them being under Main. Therefore Dates exists under Main.Mod.Dates.

Therefore I changed the JLD code to work as follows in JLD.jl / function julia_type:

function julia_type(e::Union{Symbol, Expr})
    if is_valid_type_ex(e)
        try # `try` needed to catch undefined symbols
            # `e` should be fully qualified, and thus reachable from Main
            typ = Core.eval(Main, e)
            typ == Type && return Type
            isa(typ, Type) && return typ
        catch
	    println("Mod fallback...")
	    typ = Core.eval(Main.Mod, e)
	    println("Got $typ,$(typeof(typ))")
            typ == Type && return Type
            isa(typ, Type) && return typ
			
        end
    end
    return UnsupportedType
end

Notice that I fallback in the CATCH statement to perform the Core.eval under Main.Mod as well as under Main.

A better way of fixing this, would be to pass a list of module symbols into the load method, and the code can then loop through the provided modules and attempt to Core.Evil on each provided module symbol?