JuliaDB.jl
JuliaDB.jl copied to clipboard
load("bin") broken by drop of DataType mutable field
I am trying to load a dataset from a bin directory for some out-of-core processing. However, when I run bintbl = load("bin"), I get:
ERROR: LoadError: type DataType has no field mutable
Stacktrace:
[1] getproperty
@ .\Base.jl:37 [inlined]
[2] _deser(io::Serialization.Serializer{IOStream}, t::Type)
@ JuliaDB C:\Users\mrufsvold\.julia\packages\JuliaDB\7cG1k\src\io.jl:243
[3] deserialize(io::Serialization.Serializer{IOStream}, DT::Type{JuliaDB.DIndexedTable{NamedTuple{(:datasetId, Symbol("id.patient"), Symbol("id.practice"), :V1, :V2, :V3, :V4, :V5), Tuple{String, Int64, Int64, Float64, Int64, Int64, Float64, String}}, Tuple{Int64}}})
@ JuliaDB C:\Users\mrufsvold\.julia\packages\JuliaDB\7cG1k\src\io.jl:239
[4] handle_deserialize(s::Serialization.Serializer{IOStream}, b::Int32)
@ Serialization C:\Users\mrufsvold\AppData\Local\Programs\Julia-1.7.0\share\julia\stdlib\v1.7\Serialization\src\Serialization.jl:870
[5] deserialize(s::Serialization.Serializer{IOStream})
@ Serialization C:\Users\mrufsvold\AppData\Local\Programs\Julia-1.7.0\share\julia\stdlib\v1.7\Serialization\src\Serialization.jl:801
[6] handle_deserialize(s::Serialization.Serializer{IOStream}, b::Int32)
@ Serialization C:\Users\mrufsvold\AppData\Local\Programs\Julia-1.7.0\share\julia\stdlib\v1.7\Serialization\src\Serialization.jl:907
[7] deserialize
@ C:\Users\mrufsvold\AppData\Local\Programs\Julia-1.7.0\share\julia\stdlib\v1.7\Serialization\src\Serialization.jl:801 [inlined]
[8] deserialize(s::IOStream)
@ Serialization C:\Users\mrufsvold\AppData\Local\Programs\Julia-1.7.0\share\julia\stdlib\v1.7\Serialization\src\Serialization.jl:788
[9] #198
@ C:\Users\mrufsvold\.julia\packages\JuliaDB\7cG1k\src\io.jl:176 [inlined]
[10] open(f::JuliaDB.var"#198#199", args::String; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ Base .\io.jl:330
[11] open
@ .\io.jl:328 [inlined]
[12] load(f::String; procs::Vector{Int64})
@ JuliaDB C:\Users\mrufsvold\.julia\packages\JuliaDB\7cG1k\src\io.jl:175
[13] load(f::String)
@ JuliaDB C:\Users\mrufsvold\.julia\packages\JuliaDB\7cG1k\src\io.jl:174
[14] top-level scope
@ c:\Users\mrufsvold\Projects\acic-data-challenge\src\modules\juliadb_utils.jl:9
I believe this is related to this issue noting that in Julia 1.7, DataType dropped the field mutable which broke BSON.jl.
I'm not the most advanced Julia user, so I could just be making an error myself. So I apologize if this is just me!
You should be able to change t.mutable to ismutabletype(t) at https://github.com/JuliaData/JuliaDB.jl/blob/8777d73ec944f80467980c49ad4b72fd7798040a/src/io.jl#L243.
I was able to fix this in 1.7 by doing exactly what you suggested.
I know the fix is microscopic, but I'm happy to submit a PR with this.
I also added
# Define check mutability function based on Julia version
_ver1_5orgreater = all(parse.(Ref(Int), split(string(VERSION), ".")) .>= [1,5,0])
if _ver1_5orgreater
checkmutability(t) = Base.ismutable(t)
else
checkmutability(t) = t.mutable
end
So that it will still run in older versions of Julia.
Could leave that in or cut it. Not sure how much backwards compatibility is a priority.
A PR would be much appreciated!
1.6 is the new LTS, so I would potentially just set the Julia compat bound in the Project.toml to 1.6. Also, a better way to compare versions is probably VERSION >= v"1.5".
Oh, yeah, that's a way better comparison!
I'll get the PR in shortly. Thanks for your help!