JLD.jl icon indicating copy to clipboard operation
JLD.jl copied to clipboard

BUG: cannot reconstruct tuples

Open sglyon opened this issue 9 years ago • 6 comments

When I try to read in a dataset I get the following:

ERROR: stored type Core.Tuple{Core.Float64,Core.Bool} does not match currently loaded type

I can't do it right this second, but soon I will construct a minimal example and post the code.

sglyon avatar Aug 06 '15 12:08 sglyon

I probably have the same problem

type T2
           h::Vector{Tuple{Int,Int}}
end

x = T2([(1,2),(2,3)])
@save "x.jld" x
@load "x.jld"
ERROR: MethodError: `jlconvert!` has no method matching jlconvert!(::Ptr{Tuple{Int64,Int64}}, ::Type{Tuple{Int64,Int64}}, ::JLD.JldFile, ::Ptr{UInt8})
Closest candidates are:
  jlconvert!(::Ptr{T}, ::Union{Type{Float32},Type{Float64},Type{Int16},Type{Int32},Type{Int64},Type{Int8},Type{UInt16},Type{UInt32},Type{UInt64},Type{UInt8}}, ::JLD.JldFile, ::Ptr{T})
  jlconvert!(::Ptr{T}, ::Type{Void}, ::JLD.JldFile, ::Ptr{T})
 in read_vals at /Users/pagnani/.julia/v0.4/JLD/src/JLD.jl:397
 in read_array at /Users/pagnani/.julia/v0.4/JLD/src/JLD.jl:359
 in read at /Users/pagnani/.julia/v0.4/JLD/src/JLD.jl:322
 in read_ref at /Users/pagnani/.julia/v0.4/JLD/src/JLD.jl:448
 [inlined code] from /Users/pagnani/.julia/v0.4/JLD/src/jld_types.jl:428
 in jlconvert at /Users/pagnani/.julia/v0.4/JLD/src/jld_types.jl:436
 in read_scalar at /Users/pagnani/.julia/v0.4/JLD/src/JLD.jl:348
 in read at /Users/pagnani/.julia/v0.4/JLD/src/JLD.jl:320
 in read at /Users/pagnani/.julia/v0.4/JLD/src/JLD.jl:305
 in anonymous at no file

The very same problem occurs with a

Type T1
      h::Dict{Tuple{Int,Int}, Float64}
end

Is this a known limitation of JLD?

pagnani avatar Nov 16 '15 18:11 pagnani

@pagnani Did you set INLINE_TUPLE = true? Does this happen without that change?

simonster avatar Nov 16 '15 19:11 simonster

Well the first lines of jld_types.jl are

# Controls whether tuples and non-pointerfree immutables, which Julia
# stores as references, are stored inline in compound types when
# possible. Currently this is problematic because Julia fields of these
# types may be undefined.
const INLINE_TUPLE = true

pagnani avatar Nov 16 '15 22:11 pagnani

I think I'm running into this as well. Here's my reduced case (on master of JLD.jl):

julia> immutable Foo
           t::Tuple{Float64, Float64}
       end

julia> immutable Bar
           f::Foo
       end

julia> using JLD

julia> JLD.save("test.jld", "bar", Bar(Foo((1.0, 1.0))))

julia> JLD.load("test.jld")
ERROR: stored type Foo does not match currently loaded type
 in jldatatype at /Users/jarrettrevels/.julia/v0.4/JLD/src/jld_types.jl:689
 in jldatatype at /Users/jarrettrevels/.julia/v0.4/JLD/src/jld_types.jl:679
 in read at /Users/jarrettrevels/.julia/v0.4/JLD/src/JLD.jl:370
 in read at /Users/jarrettrevels/.julia/v0.4/JLD/src/JLD.jl:346
 in anonymous at /Users/jarrettrevels/.julia/v0.4/JLD/src/JLD.jl:1185
 in jldopen at /Users/jarrettrevels/.julia/v0.4/JLD/src/JLD.jl:245
 in load at /Users/jarrettrevels/.julia/v0.4/JLD/src/JLD.jl:1184
 in load at /Users/jarrettrevels/.julia/v0.4/FileIO/src/loadsave.jl:42

This only seems to occur when the Tuple is nested within the field of another type. For example, these all work without error:

julia> JLD.save("test.jld", "foo", Foo((1.0, 1.0)))

julia> JLD.load("test.jld")
Dict{ByteString,Any} with 1 entry:
  "foo" => Foo((1.0,1.0))

julia> JLD.save("test.jld", "foo", [Foo((1.0, 1.0))])

julia> JLD.load("test.jld")
Dict{ByteString,Any} with 1 entry:
  "foo" => [Foo((1.0,1.0))]

julia> JLD.save("test.jld", "foo", Dict("f" => Foo((1.0, 1.0))))

julia> JLD.load("test.jld")
Dict{ByteString,Any} with 1 entry:
  "foo" => Dict("f"=>Foo((1.0,1.0)))

Since I'm running from this package's master branch, INLINE_TUPLE = false. I tried switching it to true just to see what would happen, and I got ERROR: Error dereferencing object when calling JLD.load("test.jld").

jrevels avatar Jan 08 '16 22:01 jrevels

In the short term, you all might be able to work around this using a custom serializer.

timholy avatar Jan 08 '16 23:01 timholy

Also hitting this. For me, it seems to have to do with what module the type is defined in:

using StaticArrays
using JLD

s = rand(SVector{3});
save("jl.jld", "s", s); 
load("jl.jld") # This works!

module M
   using StaticArrays
   immutable Typ{N,T}
       v::SVector{N,T}
   end
end

s2 = M.Typ(rand(SVector{3}));
save("jl2.jld", "s2", s2)
load("jl2.jld") # Crashes!

KristofferC avatar Nov 22 '16 14:11 KristofferC