arrow-julia icon indicating copy to clipboard operation
arrow-julia copied to clipboard

Error with CSV.Chunks and InlineStrings

Open jakkos-net opened this issue 3 years ago • 0 comments

Arrow.write("testcsv.arrow", CSV.Chunks("data.csv"), ntasks=2)

with data.csv:

heading1,heading2,heading3
test1, test2, 3
test1, test2, 3
test1, test2, 3
test1, test2, 3
...

throws this error:

Arrow.write("testcsv.arrow", CSV.Chunks("data/test/data.csv", ntasks=2))
┌ Error: error writing arrow data on partition = 2
│   exception =
│    MethodError: no method matching pointer(::String7)
│    Closest candidates are:
│      pointer(::Random.UnsafeView) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Random/src/RNGs.jl:510
│      pointer(::WeakRefString) at XXX/.julia/packages/WeakRefStrings/AN9bY/src/WeakRefStrings.jl:85
│      pointer(::WeakRefString, ::Integer) at XXX/.julia/packages/WeakRefStrings/AN9bY/src/WeakRefStrings.jl:86
│      ...
│    Stacktrace:
│      [1] pointer(A::Arrow.ToList{UInt8, true, String7, Int32}, i::Int64)
│        @ Arrow ~/.julia/packages/Arrow/x6smw/src/arraytypes/list.jl:113
│      [2] getindex
│        @ ~/.julia/packages/Arrow/x6smw/src/arraytypes/list.jl:56 [inlined]
│      [3] getindex
│        @ ~/.julia/packages/Arrow/x6smw/src/arraytypes/dictencoding.jl:48 [inlined]
│      [4] iterate
│        @ ./abstractarray.jl:1096 [inlined]
│      [5] iterate
│        @ ./abstractarray.jl:1094 [inlined]
│      [6] iterate
│        @ ./iterators.jl:159 [inlined]
│      [7] iterate
│        @ ./iterators.jl:158 [inlined]
│      [8] iterate
│        @ ./generator.jl:44 [inlined]
│      [9] Dict{String7, Int8}(kv::Base.Generator{Base.Iterators.Enumerate{Arrow.DictEncoding{String7, Int8, Arrow.List{String7, Int32, Arrow.ToList{UInt8, true, String7, Int32}}}}, Arrow.var"#52#54"})
│        @ Base ./dict.jl:103
...

jakkos-net avatar Dec 30 '21 15:12 jakkos-net