"SymbolServer not defined" error when called within different module

[Neelo16]

When SymbolServer is used within a module that imports SymbolServer, by a different module that does not import SymbolServer, SymbolServer is stuck on an infinite loop, re-caching packages. If we rethrow the exception on SymbolServer.jl, we can see that it always throws UndefVarError: SymbolServer not defined, unless we import SymbolServer into the calling module. This can be reproduced with the following code:

module Foo

using SymbolServer

function bar()
    sserver = SymbolServerProcess()
    SymbolServer.getstore(sserver)
end

end

Foo.bar()

If we add using SymbolServer outside the Foo module, we get the expected behavior, and all packages are succesfully cached and loaded:

using SymbolServer

module Foo

using SymbolServer

function bar()
    sserver = SymbolServerProcess()
    SymbolServer.getstore(sserver)
end

end

Foo.bar()

[davidanthoff]

master now has what is essentially a complete rewrite. I'm going to close this for now, if this is still an issue, please reopen!

[non-Jedi]

This is still an issue and should be reopened. runing a LanguageServerInstance in a scope without using SymbolServer throws an UndefVarError:

ERROR: UndefVarError: SymbolServer not defined
Stacktrace:
 [1] deserialize_module(::Serialization.Serializer{IOStream}) at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:915
 [2] handle_deserialize(::Serialization.Serializer{IOStream}, ::Int32) at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:812
 [3] deserialize(::Serialization.Serializer{IOStream}) at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:735
 [4] deserialize_datatype(::Serialization.Serializer{IOStream}, ::Bool) at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:1210
 [5] handle_deserialize(::Serialization.Serializer{IOStream}, ::Int32) at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:788
 [6] deserialize(::Serialization.Serializer{IOStream}) at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:735
 [7] handle_deserialize(::Serialization.Serializer{IOStream}, ::Int32) at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:795
 [8] deserialize(::Serialization.Serializer{IOStream}) at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:735
 [9] handle_deserialize(::Serialization.Serializer{IOStream}, ::Int32) at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:838
 [10] deserialize(::Serialization.Serializer{IOStream}) at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:735
 [11] deserialize at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:722 [inlined]
 [12] #25 at /home/adam/.julia/packages/SymbolServer/vTGsB/src/SymbolServer.jl:171 [inlined]
 [13] open(::SymbolServer.var"#25#26", ::String; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at ./io.jl:298
 [14] open at ./io.jl:296 [inlined]
 [15] load_package_from_cache_into_store!(::SymbolServer.SymbolServerInstance, ::Base.UUID, ::Dict{Base.UUID,Pkg.Types.PackageEntry}, ::Dict{Symbol,SymbolServer.ModuleStore}) at /home/adam/.julia/packages/SymbolServer/vTGsB/src/SymbolServer.jl:170
 [16] load_project_packages_into_store!(::SymbolServer.SymbolServerInstance, ::String, ::Dict{Symbol,SymbolServer.ModuleStore}) at /home/adam/.julia/packages/SymbolServer/vTGsB/src/SymbolServer.jl:142
 [17] getstore(::SymbolServer.SymbolServerInstance, ::String, ::LanguageServer.var"#2#4"{LanguageServerInstance}, ::Nothing) at /home/adam/.julia/packages/SymbolServer/vTGsB/src/SymbolServer.jl:100
 [18] macro expansion at /home/adam/.julia/packages/LanguageServer/mpNvN/src/languageserverinstance.jl:157 [inlined]
 [19] (::LanguageServer.var"#1#3"{LanguageServerInstance})() at ./task.jl:358
┌ Warning: Tried to load Sodium but failed to load from disc, re-caching.
└ @ SymbolServer ~/.julia/packages/SymbolServer/vTGsB/src/SymbolServer.jl:179
ERROR: UndefVarError: SymbolServer not defined
Stacktrace:
 [1] deserialize_module(::Serialization.Serializer{IOStream}) at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:915
 [2] handle_deserialize(::Serialization.Serializer{IOStream}, ::Int32) at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:812
 [3] deserialize(::Serialization.Serializer{IOStream}) at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:735
 [4] deserialize_datatype(::Serialization.Serializer{IOStream}, ::Bool) at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:1210
 [5] handle_deserialize(::Serialization.Serializer{IOStream}, ::Int32) at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:788
 [6] deserialize(::Serialization.Serializer{IOStream}) at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:735
 [7] handle_deserialize(::Serialization.Serializer{IOStream}, ::Int32) at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:795
 [8] deserialize(::Serialization.Serializer{IOStream}) at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:735
 [9] handle_deserialize(::Serialization.Serializer{IOStream}, ::Int32) at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:838
 [10] deserialize(::Serialization.Serializer{IOStream}) at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:735
 [11] deserialize at /builddir/julia-1.4.1/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:722 [inlined]
 [12] #25 at /home/adam/.julia/packages/SymbolServer/vTGsB/src/SymbolServer.jl:171 [inlined]
 [13] open(::SymbolServer.var"#25#26", ::String; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at ./io.jl:298
 [14] open at ./io.jl:296 [inlined]
 [15] load_package_from_cache_into_store!(::SymbolServer.SymbolServerInstance, ::Base.UUID, ::Dict{Base.UUID,Pkg.Types.PackageEntry}, ::Dict{Symbol,SymbolServer.ModuleStore}) at /home/adam/.julia/packages/SymbolServer/vTGsB/src/SymbolServer.jl:170
 [16] load_project_packages_into_store!(::SymbolServer.SymbolServerInstance, ::String, ::Dict{Symbol,SymbolServer.ModuleStore}) at /home/adam/.julia/packages/SymbolServer/vTGsB/src/SymbolServer.jl:142
 [17] getstore(::SymbolServer.SymbolServerInstance, ::String, ::LanguageServer.var"#2#4"{LanguageServerInstance}, ::Nothing) at /home/adam/.julia/packages/SymbolServer/vTGsB/src/SymbolServer.jl:100
 [18] macro expansion at /home/adam/.julia/packages/LanguageServer/mpNvN/src/languageserverinstance.jl:157 [inlined]
 [19] (::LanguageServer.var"#1#3"{LanguageServerInstance})() at ./task.jl:358
┌ Warning: Tried to load YAML but failed to load from disc, re-caching.

[davidanthoff]

My vague hope is that this is somehow related to serialization (which we are using), and that once we move to our own format for the cache, this problem will miraculously disappear :)

[davidanthoff]

I thought a bit more about this, and I'm fairly positive that the following explains what is going on:

In the symbol server process, we are serializing things onto disc using Julia serialization. When we deserialize in the LS process, it probably is looking for the types that should be materialized in the Main.SymbolServer module, and not in Main.LanguageServer.SymbolServer.

I think the proper way to fix this is to move to our own custom format rather sooner than later, i.e. get https://github.com/julia-vscode/SymbolServer.jl/pull/155 ready for merging.