BSON.jl icon indicating copy to clipboard operation
BSON.jl copied to clipboard

UndefRefError: access to undefined reference

Open tk3369 opened this issue 7 years ago • 14 comments

Nice package, Mike! I found this obscure error message when saving this custom AbstractArray type from SASLib.jl.

julia> using SASLib

julia> p = SASLib.ObjectPool{String,UInt8}("", 3)
3-element SASLib.ObjectPool{String,UInt8}:
 ""
 ""
 ""

julia> BSON.@save "test.bson" p
ERROR: UndefRefError: access to undefined reference
Stacktrace:
 [1] lower(::Array{String,1}) at /Users/tomkwong/.julia/v0.6/BSON/src/extensions.jl:60
 [2] _lower_recursive(::Array{String,1}, ::ObjectIdDict, ::Array{Any,1}) at /Users/tomkwong/.julia/v0.6/BSON/src/write.jl:62
 [3] applychildren!(::BSON.##7#11{ObjectIdDict,Array{Any,1}}, ::Array{Any,1}) at /Users/tomkwong/.julia/v0.6/BSON/src/BSON.jl:25
 [4] _lower_recursive(::Array{Any,1}, ::ObjectIdDict, ::Array{Any,1}) at /Users/tomkwong/.julia/v0.6/BSON/src/write.jl:62
 [5] applychildren!(::BSON.##7#11{ObjectIdDict,Array{Any,1}}, ::Dict{Symbol,Any}) at /Users/tomkwong/.julia/v0.6/BSON/src/BSON.jl:18
 [6] _lower_recursive(::Dict{String,UInt8}, ::ObjectIdDict, ::Array{Any,1}) at /Users/tomkwong/.julia/v0.6/BSON/src/write.jl:62
 [7] applychildren!(::BSON.##7#11{ObjectIdDict,Array{Any,1}}, ::Array{Any,1}) at /Users/tomkwong/.julia/v0.6/BSON/src/BSON.jl:25
 [8] _lower_recursive(::Array{Any,1}, ::ObjectIdDict, ::Array{Any,1}) at /Users/tomkwong/.julia/v0.6/BSON/src/write.jl:62
 [9] applychildren!(::BSON.##7#11{ObjectIdDict,Array{Any,1}}, ::Dict{Symbol,Any}) at /Users/tomkwong/.julia/v0.6/BSON/src/BSON.jl:18
 [10] _lower_recursive(::SASLib.ObjectPool{String,UInt8}, ::ObjectIdDict, ::Array{Any,1}) at /Users/tomkwong/.julia/v0.6/BSON/src/write.jl:62
 [11] applychildren!(::BSON.##7#11{ObjectIdDict,Array{Any,1}}, ::Dict{Symbol,Any}) at /Users/tomkwong/.julia/v0.6/BSON/src/BSON.jl:18
 [12] _lower_recursive(::Dict{Symbol,SASLib.ObjectPool{String,UInt8}}, ::ObjectIdDict, ::Array{Any,1}) at /Users/tomkwong/.julia/v0.6/BSON/src/write.jl:62
 [13] lower_recursive(::Dict{Symbol,SASLib.ObjectPool{String,UInt8}}) at /Users/tomkwong/.julia/v0.6/BSON/src/write.jl:73
 [14] open(::BSON.##16#17{Dict{Symbol,SASLib.ObjectPool{String,UInt8}}}, ::String, ::String) at ./iostream.jl:152
 [15] bson(::String, ::Dict{Symbol,SASLib.ObjectPool{String,UInt8}}) at /Users/tomkwong/.julia/v0.6/BSON/src/write.jl:83

tk3369 avatar Apr 07 '18 17:04 tk3369

Any chance you can narrow this down a bit? If you can figure out the minimal type definition that triggers this, it should be easy to fix.

MikeInnes avatar Apr 18 '18 15:04 MikeInnes

Looks like it fails for all Dict with non-symbol keys:

julia> c4 = Dict("a" => :b)
Dict{String,Symbol} with 1 entry:
  "a" => :b

julia> BSON.@save "test.bson" c4
ERROR: UndefRefError: access to undefined reference
Stacktrace:
 [1] lower(::Array{String,1}) at /Users/tomkwong/.julia/v0.6/BSON/src/extensions.jl:60
 [2] _lower_recursive(::Array{String,1}, ::ObjectIdDict, ::Array{Any,1}) at /Users/tomkwong/.julia/v0.6/BSON/src/write.jl:62
 [3] applychildren!(::BSON.##7#11{ObjectIdDict,Array{Any,1}}, ::Array{Any,1}) at /Users/tomkwong/.julia/v0.6/BSON/src/BSON.jl:25
 [4] _lower_recursive(::Array{Any,1}, ::ObjectIdDict, ::Array{Any,1}) at /Users/tomkwong/.julia/v0.6/BSON/src/write.jl:62
 [5] applychildren!(::BSON.##7#11{ObjectIdDict,Array{Any,1}}, ::Dict{Symbol,Any}) at /Users/tomkwong/.julia/v0.6/BSON/src/BSON.jl:18
 [6] _lower_recursive(::Dict{String,Symbol}, ::ObjectIdDict, ::Array{Any,1}) at /Users/tomkwong/.julia/v0.6/BSON/src/write.jl:62
 [7] applychildren!(::BSON.##7#11{ObjectIdDict,Array{Any,1}}, ::Dict{Symbol,Any}) at /Users/tomkwong/.julia/v0.6/BSON/src/BSON.jl:18
 [8] _lower_recursive(::Dict{Symbol,Dict{String,Symbol}}, ::ObjectIdDict, ::Array{Any,1}) at /Users/tomkwong/.julia/v0.6/BSON/src/write.jl:62
 [9] lower_recursive(::Dict{Symbol,Dict{String,Symbol}}) at /Users/tomkwong/.julia/v0.6/BSON/src/write.jl:73
 [10] open(::BSON.##16#17{Dict{Symbol,Dict{String,Symbol}}}, ::String, ::String) at ./iostream.jl:152
 [11] bson(::String, ::Dict{Symbol,Dict{String,Symbol}}) at /Users/tomkwong/.julia/v0.6/BSON/src/write.jl:83

julia> c4 = Dict("a" => 1)
Dict{String,Int64} with 1 entry:
  "a" => 1

julia> BSON.@save "test.bson" c4
ERROR: UndefRefError: access to undefined reference

julia> c4 = Dict(1 => "a")
Dict{Int64,String} with 1 entry:
  1 => "a"

julia> BSON.@save "test.bson" c4
ERROR: UndefRefError: access to undefined reference

tk3369 avatar Apr 20 '18 04:04 tk3369

This is really interesting. It doesn't fail for all non-symbol keys; for example, it works for integer keys/integer values.

Key type Value type Does it work?
Symbol Symbol Yes
Symbol String Yes
Symbol Int64 Yes
String Symbol No
String String No
String Int64 No
Int64 Symbol No
Int64 String No
Int64 Int64 Yes
julia> x1 = Dict(:a => :a); BSON.@save "test.bson" x1

julia> x2= Dict(:a => "a"); BSON.@save "test.bson" x2

julia> x3= Dict(:a => 1); BSON.@save "test.bson" x3

julia> x4 = Dict("a" => :a); BSON.@save "test.bson" x4
ERROR: UndefRefError: access to undefined reference
Stacktrace:
 [1] lower(::Array{String,1}) at /Users/dilum/.julia/v0.6/BSON/src/extensions.jl:60
 [2] _lower_recursive(::Array{String,1}, ::ObjectIdDict, ::Array{Any,1}) at /Users/dilum/.julia/v0.6/BSON/src/write.jl:62
 [3] applychildren!(::BSON.##7#11{ObjectIdDict,Array{Any,1}}, ::Array{Any,1}) at /Users/dilum/.julia/v0.6/BSON/src/BSON.jl:25
 [4] _lower_recursive(::Array{Any,1}, ::ObjectIdDict, ::Array{Any,1}) at /Users/dilum/.julia/v0.6/BSON/src/write.jl:62
 [5] applychildren!(::BSON.##7#11{ObjectIdDict,Array{Any,1}}, ::Dict{Symbol,Any}) at /Users/dilum/.julia/v0.6/BSON/src/BSON.jl:18
 [6] _lower_recursive(::Dict{String,Symbol}, ::ObjectIdDict, ::Array{Any,1}) at /Users/dilum/.julia/v0.6/BSON/src/write.jl:62
 [7] applychildren!(::BSON.##7#11{ObjectIdDict,Array{Any,1}}, ::Dict{Symbol,Any}) at /Users/dilum/.julia/v0.6/BSON/src/BSON.jl:18
 [8] _lower_recursive(::Dict{Symbol,Dict{String,Symbol}}, ::ObjectIdDict, ::Array{Any,1}) at /Users/dilum/.julia/v0.6/BSON/src/write.jl:62
 [9] lower_recursive(::Dict{Symbol,Dict{String,Symbol}}) at /Users/dilum/.julia/v0.6/BSON/src/write.jl:73
 [10] open(::BSON.##16#17{Dict{Symbol,Dict{String,Symbol}}}, ::String, ::String) at ./iostream.jl:152
 [11] bson(::String, ::Dict{Symbol,Dict{String,Symbol}}) at /Users/dilum/.julia/v0.6/BSON/src/write.jl:83

julia> x5 = Dict("a" => "a"); BSON.@save "test.bson" x5
ERROR: UndefRefError: access to undefined reference
Stacktrace:

julia> x6 = Dict("a" => 1); BSON.@save "test.bson" x6
ERROR: UndefRefError: access to undefined reference
Stacktrace:

julia> x7 = Dict(1 => :a); BSON.@save "test.bson" x7
ERROR: UndefRefError: access to undefined reference
Stacktrace:

julia> x8 = Dict(1 => "a"); BSON.@save "test.bson" x8
ERROR: UndefRefError: access to undefined reference
Stacktrace:

julia> x9 = Dict(1 => 1); BSON.@save "test.bson" x9

julia>

DilumAluthge avatar May 16 '18 05:05 DilumAluthge

This works:

julia> BSON.bson("test.bson", Dict(:a => 1, :b => Dict(:c => 3, :d => Dict(:e => 5) )  ))

julia>

And this works:

julia> BSON.bson("test.bson", Dict(:a => 1, :b => Dict(:c => 3, :d => Dict(5 => 5) )  ))

julia>

And this also works:

julia> BSON.bson("test.bson", Dict(:a => 1, :b => Dict(:c => 3, :d => Dict(1.23 => 5) )  ))

julia>

But this does NOT work:

julia> BSON.bson("test.bson", Dict(:a => 1, :b => Dict(:c => 3, :d => Dict("e" => 5) )  ))
ERROR: UndefRefError: access to undefined reference
Stacktrace:
 [1] lower(::Array{String,1}) at /Users/dilum/.julia/v0.6/BSON/src/extensions.jl:60
 [2] _lower_recursive(::Array{String,1}, ::ObjectIdDict, ::Array{Any,1}) at /Users/dilum/.julia/v0.6/BSON/src/write.jl:62
 [3] applychildren!(::BSON.##7#11{ObjectIdDict,Array{Any,1}}, ::Array{Any,1}) at /Users/dilum/.julia/v0.6/BSON/src/BSON.jl:25
 [4] _lower_recursive(::Array{Any,1}, ::ObjectIdDict, ::Array{Any,1}) at /Users/dilum/.julia/v0.6/BSON/src/write.jl:62
 [5] applychildren!(::BSON.##7#11{ObjectIdDict,Array{Any,1}}, ::Dict{Symbol,Any}) at /Users/dilum/.julia/v0.6/BSON/src/BSON.jl:18
 [6] _lower_recursive(::Dict{String,Int64}, ::ObjectIdDict, ::Array{Any,1}) at /Users/dilum/.julia/v0.6/BSON/src/write.jl:62
 [7] applychildren!(::BSON.##7#11{ObjectIdDict,Array{Any,1}}, ::Dict{Symbol,Any}) at /Users/dilum/.julia/v0.6/BSON/src/BSON.jl:18
 [8] _lower_recursive(::Dict{Symbol,Any}, ::ObjectIdDict, ::Array{Any,1}) at /Users/dilum/.julia/v0.6/BSON/src/write.jl:62
 [9] applychildren!(::BSON.##7#11{ObjectIdDict,Array{Any,1}}, ::Dict{Symbol,Any}) at /Users/dilum/.julia/v0.6/BSON/src/BSON.jl:18
 [10] _lower_recursive(::Dict{Symbol,Any}, ::ObjectIdDict, ::Array{Any,1}) at /Users/dilum/.julia/v0.6/BSON/src/write.jl:62
 [11] lower_recursive(::Dict{Symbol,Any}) at /Users/dilum/.julia/v0.6/BSON/src/write.jl:73
 [12] open(::BSON.##16#17{Dict{Symbol,Any}}, ::String, ::String) at ./iostream.jl:152
 [13] bson(::String, ::Dict{Symbol,Any}) at /Users/dilum/.julia/v0.6/BSON/src/write.jl:83

julia>

DilumAluthge avatar May 26 '18 01:05 DilumAluthge

So it looks like you can't have a dict with non-symbol keys anywhere inside the object you are trying to save - with the very weird exception of numerical keys and numerical values.

DilumAluthge avatar May 26 '18 01:05 DilumAluthge

~~I've found a hacky way to solve this issue - see PR #6.~~

DilumAluthge avatar May 26 '18 02:05 DilumAluthge

Disregard my previous comment - see better PR here: #7

DilumAluthge avatar May 31 '18 22:05 DilumAluthge

The following code produces this error, too.

using StochasticDiffEq, BSON

function lorenz(du,u,p,t)
  du[1] = 10.0(u[2]-u[1])
  du[2] = u[1]*(28.0-u[3]) - u[2]
  du[3] = u[1]*u[2] - (8/3)*u[3]
end

function σ_lorenz(du,u,p,t)
  du[1] = 3.0
  du[2] = 3.0
  du[3] = 3.0
end

prob_sde_lorenz = SDEProblem(lorenz,σ_lorenz,[1.0,0.0,0.0],(0.0,10.0))
sol = solve(prob_sde_lorenz, SOSRA2())
bson("/tmp/test.bson",Dict(:sol=>sol))
julia> bson("/tmp/test.bson",Dict(:sol=>sol))
ERROR: UndefRefError: access to undefined reference
Stacktrace:
 [1] lower at /home/scheme/.julia/packages/BSON/kxdIr/src/extensions.jl:66 [inlined]
 [2] _lower_recursive(::Array{Tuple{Float64,Array{Float64,1},Array{Float64,1}},1}, ::IdDict{Any,Any}, ::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [3] (::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}})(::Array{Tuple{Float64,Array{Float64,1},Array{Float64,1}},1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [4] applychildren!(::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}}, ::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/BSON.jl:28
 [5] _lower_recursive(::Array{Any,1}, ::IdDict{Any,Any}, ::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [6] (::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}})(::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [7] applychildren!(::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}}, ::Dict{Symbol,Any}) at /home/scheme/.julia/packages/BSON/kxdIr/src/BSON.jl:21
 [8] _lower_recursive(::DataStructures.DequeBlock{Tuple{Float64,Array{Float64,1},Array{Float64,1}}}, ::IdDict{Any,Any}, ::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [9] (::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}})(::DataStructures.DequeBlock{Tuple{Float64,Array{Float64,1},Array{Float64,1}}}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [10] applychildren!(::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}}, ::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/BSON.jl:28
 [11] _lower_recursive(::Array{Any,1}, ::IdDict{Any,Any}, ::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [12] (::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}})(::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [13] applychildren!(::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}}, ::Dict{Symbol,Any}) at /home/scheme/.julia/packages/BSON/kxdIr/src/BSON.jl:21
 [14] _lower_recursive(::DataStructures.Deque{Tuple{Float64,Array{Float64,1},Array{Float64,1}}}, ::IdDict{Any,Any}, ::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [15] (::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}})(::DataStructures.Deque{Tuple{Float64,Array{Float64,1},Array{Float64,1}}}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [16] applychildren!(::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}}, ::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/BSON.jl:28
 [17] _lower_recursive(::Array{Any,1}, ::IdDict{Any,Any}, ::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [18] (::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}})(::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [19] applychildren!(::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}}, ::Dict{Symbol,Any}) at /home/scheme/.julia/packages/BSON/kxdIr/src/BSON.jl:21
 [20] _lower_recursive(::DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}}, ::IdDict{Any,Any}, ::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [21] (::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}})(::DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [22] applychildren!(::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}}, ::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/BSON.jl:28
 [23] _lower_recursive(::Array{Any,1}, ::IdDict{Any,Any}, ::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [24] (::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}})(::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [25] applychildren!(::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}}, ::Dict{Symbol,Any}) at /home/scheme/.julia/packages/BSON/kxdIr/src/BSON.jl:21
 [26] _lower_recursive(::DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},typeof(DiffEqNoiseProcess.INPLACE_WHITE_NOISE_DIST),typeof(DiffEqNoiseProcess.INPLACE_WHITE_NOISE_BRIDGE),true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus}, ::IdDict{Any,Any}, ::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [27] (::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}})(::DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},typeof(DiffEqNoiseProcess.INPLACE_WHITE_NOISE_DIST),typeof(DiffEqNoiseProcess.INPLACE_WHITE_NOISE_BRIDGE),true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [28] applychildren!(::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}}, ::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/BSON.jl:28
 [29] _lower_recursive(::Array{Any,1}, ::IdDict{Any,Any}, ::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [30] (::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}})(::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [31] applychildren!(::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}}, ::Dict{Symbol,Any}) at /home/scheme/.julia/packages/BSON/kxdIr/src/BSON.jl:21
 [32] _lower_recursive(::RODESolution{Float64,2,Array{Array{Float64,1},1},Nothing,Nothing,Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},typeof(DiffEqNoiseProcess.INPLACE_WHITE_NOISE_DIST),typeof(DiffEqNoiseProcess.INPLACE_WHITE_NOISE_BRIDGE),true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},SDEProblem{Array{Float64,1},Tuple{Float64,Float64},true,Nothing,Nothing,SDEFunction{true,typeof(lorenz),typeof(σ_lorenz),UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},typeof(σ_lorenz),Nothing,Nothing},SOSRA2,StochasticDiffEq.LinearInterpolationData{Array{Array{Float64,1},1},Array{Float64,1}}}, ::IdDict{Any,Any}, ::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [33] (::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}})(::RODESolution{Float64,2,Array{Array{Float64,1},1},Nothing,Nothing,Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},typeof(DiffEqNoiseProcess.INPLACE_WHITE_NOISE_DIST),typeof(DiffEqNoiseProcess.INPLACE_WHITE_NOISE_BRIDGE),true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},SDEProblem{Array{Float64,1},Tuple{Float64,Float64},true,Nothing,Nothing,SDEFunction{true,typeof(lorenz),typeof(σ_lorenz),UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},typeof(σ_lorenz),Nothing,Nothing},SOSRA2,StochasticDiffEq.LinearInterpolationData{Array{Array{Float64,1},1},Array{Float64,1}}}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [34] applychildren!(::getfield(BSON, Symbol("##7#11")){IdDict{Any,Any},Array{Any,1}}, ::Dict{Symbol,Any}) at /home/scheme/.julia/packages/BSON/kxdIr/src/BSON.jl:21
 [35] _lower_recursive(::Dict{Symbol,RODESolution{Float64,2,Array{Array{Float64,1},1},Nothing,Nothing,Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},typeof(DiffEqNoiseProcess.INPLACE_WHITE_NOISE_DIST),typeof(DiffEqNoiseProcess.INPLACE_WHITE_NOISE_BRIDGE),true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},SDEProblem{Array{Float64,1},Tuple{Float64,Float64},true,Nothing,Nothing,SDEFunction{true,typeof(lorenz),typeof(σ_lorenz),UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},typeof(σ_lorenz),Nothing,Nothing},SOSRA2,StochasticDiffEq.LinearInterpolationData{Array{Array{Float64,1},1},Array{Float64,1}}}}, ::IdDict{Any,Any}, ::Array{Any,1}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:62
 [36] lower_recursive(::Dict{Symbol,RODESolution{Float64,2,Array{Array{Float64,1},1},Nothing,Nothing,Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},typeof(DiffEqNoiseProcess.INPLACE_WHITE_NOISE_DIST),typeof(DiffEqNoiseProcess.INPLACE_WHITE_NOISE_BRIDGE),true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},SDEProblem{Array{Float64,1},Tuple{Float64,Float64},true,Nothing,Nothing,SDEFunction{true,typeof(lorenz),typeof(σ_lorenz),UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},typeof(σ_lorenz),Nothing,Nothing},SOSRA2,StochasticDiffEq.LinearInterpolationData{Array{Array{Float64,1},1},Array{Float64,1}}}}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:73
 [37] bson(::IOStream, ::Dict{Symbol,RODESolution{Float64,2,Array{Array{Float64,1},1},Nothing,Nothing,Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},typeof(DiffEqNoiseProcess.INPLACE_WHITE_NOISE_DIST),typeof(DiffEqNoiseProcess.INPLACE_WHITE_NOISE_BRIDGE),true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},SDEProblem{Array{Float64,1},Tuple{Float64,Float64},true,Nothing,Nothing,SDEFunction{true,typeof(lorenz),typeof(σ_lorenz),UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},typeof(σ_lorenz),Nothing,Nothing},SOSRA2,StochasticDiffEq.LinearInterpolationData{Array{Array{Float64,1},1},Array{Float64,1}}}}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:81
 [38] #14 at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:83 [inlined]
 [39] #open#292(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::getfield(BSON, Symbol("##14#15")){Dict{Symbol,RODESolution{Float64,2,Array{Array{Float64,1},1},Nothing,Nothing,Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},typeof(DiffEqNoiseProcess.INPLACE_WHITE_NOISE_DIST),typeof(DiffEqNoiseProcess.INPLACE_WHITE_NOISE_BRIDGE),true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},SDEProblem{Array{Float64,1},Tuple{Float64,Float64},true,Nothing,Nothing,SDEFunction{true,typeof(lorenz),typeof(σ_lorenz),UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},typeof(σ_lorenz),Nothing,Nothing},SOSRA2,StochasticDiffEq.LinearInterpolationData{Array{Array{Float64,1},1},Array{Float64,1}}}}}, ::String, ::Vararg{String,N} where N) at ./iostream.jl:369
 [40] open at ./iostream.jl:367 [inlined]
 [41] bson(::String, ::Dict{Symbol,RODESolution{Float64,2,Array{Array{Float64,1},1},Nothing,Nothing,Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},typeof(DiffEqNoiseProcess.INPLACE_WHITE_NOISE_DIST),typeof(DiffEqNoiseProcess.INPLACE_WHITE_NOISE_BRIDGE),true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},SDEProblem{Array{Float64,1},Tuple{Float64,Float64},true,Nothing,Nothing,SDEFunction{true,typeof(lorenz),typeof(σ_lorenz),UniformScaling{Bool},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},typeof(σ_lorenz),Nothing,Nothing},SOSRA2,StochasticDiffEq.LinearInterpolationData{Array{Array{Float64,1},1},Array{Float64,1}}}}) at /home/scheme/.julia/packages/BSON/kxdIr/src/write.jl:83
 [42] top-level scope at none:0

julia>

YingboMa avatar Oct 26 '18 22:10 YingboMa

The same error occurs when saving an optimizer in Flux: https://github.com/FluxML/Flux.jl/issues/737

As far as I can see, the underlying problem is that it is not possible to save an IdDict.

devmotion avatar Jun 17 '19 14:06 devmotion

Actually, the underlying problem here is that currently BSON.jl cannot handle arrays with undefined values. The issue for tracking this is: https://github.com/MikeInnes/BSON.jl/issues/43

Not being able to save certain Dicts is just a symptom of https://github.com/MikeInnes/BSON.jl/issues/43. The underlying implementation of Dict means that Dicts will often contain (under the hood) one or more Arrays with undefined values. In order to save/load the Dict, you need to be able to save/load the Arrays.

Once https://github.com/MikeInnes/BSON.jl/issues/43 is solved, there will be no problem saving/loading any kind of Dict.

DilumAluthge avatar Jun 17 '19 14:06 DilumAluthge

examples in https://github.com/JuliaIO/BSON.jl/issues/3#issuecomment-392226367 all work now, this should be closed

CarloLucibello avatar Apr 21 '20 10:04 CarloLucibello

examples in #3 (comment) all work now, this should be closed

Before we close this issue, I think that we should take the examples from that issue and add them to the test suite of this package.

DilumAluthge avatar Apr 21 '20 18:04 DilumAluthge

I note that this error also happens in the absence of undef entries. I ran into this with a dataframe column type

SentinelArrays.SentinelArray{String,1,UndefInitializer,Missing,Array{String,1}})

see https://github.com/JuliaData/SentinelArrays.jl/issues/24#issuecomment-667907257

kafisatz avatar Aug 03 '20 09:08 kafisatz

Also getting this error using the AbstractFFTs package.

using AbstractFFTs

a = FFTW.plan_fft(Array{Complex{Float64}}(undef,1))

var1 = Dict(:fft => a)

path = "_research/tmp/"
mkpath(path)

bson(path*"1.bson",var1)
typeof(a) = FFTW.cFFTWPlan{Complex{Float64},-1,false,1,UnitRange{Int64}}

I'm fairly new to Julia, so I'm not sure If there's more information I can provide to help pinpoint how/where this error is occurring.

umbriquse avatar Dec 07 '20 19:12 umbriquse