JuliaDB.jl icon indicating copy to clipboard operation
JuliaDB.jl copied to clipboard

load("bin") broken by drop of DataType mutable field

Open mrufsvold opened this issue 3 years ago • 4 comments

I am trying to load a dataset from a bin directory for some out-of-core processing. However, when I run bintbl = load("bin"), I get:

ERROR: LoadError: type DataType has no field mutable
Stacktrace:
  [1] getproperty
    @ .\Base.jl:37 [inlined]
  [2] _deser(io::Serialization.Serializer{IOStream}, t::Type)
    @ JuliaDB C:\Users\mrufsvold\.julia\packages\JuliaDB\7cG1k\src\io.jl:243
  [3] deserialize(io::Serialization.Serializer{IOStream}, DT::Type{JuliaDB.DIndexedTable{NamedTuple{(:datasetId, Symbol("id.patient"), Symbol("id.practice"), :V1, :V2, :V3, :V4, :V5), Tuple{String, Int64, Int64, Float64, Int64, Int64, Float64, String}}, Tuple{Int64}}})
    @ JuliaDB C:\Users\mrufsvold\.julia\packages\JuliaDB\7cG1k\src\io.jl:239
  [4] handle_deserialize(s::Serialization.Serializer{IOStream}, b::Int32)
    @ Serialization C:\Users\mrufsvold\AppData\Local\Programs\Julia-1.7.0\share\julia\stdlib\v1.7\Serialization\src\Serialization.jl:870
  [5] deserialize(s::Serialization.Serializer{IOStream})
    @ Serialization C:\Users\mrufsvold\AppData\Local\Programs\Julia-1.7.0\share\julia\stdlib\v1.7\Serialization\src\Serialization.jl:801
  [6] handle_deserialize(s::Serialization.Serializer{IOStream}, b::Int32)
    @ Serialization C:\Users\mrufsvold\AppData\Local\Programs\Julia-1.7.0\share\julia\stdlib\v1.7\Serialization\src\Serialization.jl:907
  [7] deserialize
    @ C:\Users\mrufsvold\AppData\Local\Programs\Julia-1.7.0\share\julia\stdlib\v1.7\Serialization\src\Serialization.jl:801 [inlined]    
  [8] deserialize(s::IOStream)
    @ Serialization C:\Users\mrufsvold\AppData\Local\Programs\Julia-1.7.0\share\julia\stdlib\v1.7\Serialization\src\Serialization.jl:788
  [9] #198
    @ C:\Users\mrufsvold\.julia\packages\JuliaDB\7cG1k\src\io.jl:176 [inlined]
 [10] open(f::JuliaDB.var"#198#199", args::String; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Base .\io.jl:330
 [11] open
    @ .\io.jl:328 [inlined]
 [12] load(f::String; procs::Vector{Int64})
    @ JuliaDB C:\Users\mrufsvold\.julia\packages\JuliaDB\7cG1k\src\io.jl:175
 [13] load(f::String)
    @ JuliaDB C:\Users\mrufsvold\.julia\packages\JuliaDB\7cG1k\src\io.jl:174
 [14] top-level scope
    @ c:\Users\mrufsvold\Projects\acic-data-challenge\src\modules\juliadb_utils.jl:9

I believe this is related to this issue noting that in Julia 1.7, DataType dropped the field mutable which broke BSON.jl.

I'm not the most advanced Julia user, so I could just be making an error myself. So I apologize if this is just me!

mrufsvold avatar Feb 22 '22 18:02 mrufsvold

You should be able to change t.mutable to ismutabletype(t) at https://github.com/JuliaData/JuliaDB.jl/blob/8777d73ec944f80467980c49ad4b72fd7798040a/src/io.jl#L243.

jpsamaroo avatar Feb 22 '22 19:02 jpsamaroo

I was able to fix this in 1.7 by doing exactly what you suggested.

I know the fix is microscopic, but I'm happy to submit a PR with this.

I also added

# Define check mutability function based on Julia version
_ver1_5orgreater = all(parse.(Ref(Int), split(string(VERSION), ".")) .>= [1,5,0])
if _ver1_5orgreater
    checkmutability(t) = Base.ismutable(t) 
else
    checkmutability(t) = t.mutable
end

So that it will still run in older versions of Julia.

Could leave that in or cut it. Not sure how much backwards compatibility is a priority.

mrufsvold avatar Feb 22 '22 20:02 mrufsvold

A PR would be much appreciated!

1.6 is the new LTS, so I would potentially just set the Julia compat bound in the Project.toml to 1.6. Also, a better way to compare versions is probably VERSION >= v"1.5".

jpsamaroo avatar Feb 22 '22 20:02 jpsamaroo

Oh, yeah, that's a way better comparison!

I'll get the PR in shortly. Thanks for your help!

mrufsvold avatar Feb 22 '22 20:02 mrufsvold