StructArrays.jl icon indicating copy to clipboard operation
StructArrays.jl copied to clipboard

push! does not check for column length consistency

Open bkamins opened this issue 5 years ago • 1 comments

Are these two scenarios intended?

julia> t = StructArray([(a=1,b=2)])
1-element StructArray(::Array{Int64,1}, ::Array{Int64,1}) with eltype NamedTuple{(:a, :b),Tuple{Int64,Int64}}:
 (a = 1, b = 2)

julia> push!(t,(a=2,b="3"))
ERROR: MethodError: Cannot `convert` an object of type String to an object of type Int64

julia> t
2-element StructArray(::Array{Int64,1}, ::Array{Int64,1}) with eltype NamedTuple{(:a, :b),Tuple{Int64,Int64}}:
 (a = 1, b = 2)
 (a = 2, b = 140572479985616)


julia> t = StructArray([(a=1,b="2")])
1-element StructArray(::Array{Int64,1}, ::Array{String,1}) with eltype NamedTuple{(:a, :b),Tuple{Int64,String}}:
 (a = 1, b = "2")

julia> push!(t,(a=2,b=3))
ERROR: MethodError: Cannot `convert` an object of type Int64 to an object of type String

julia> t
2-element StructArray(::Array{Int64,1}, ::Array{String,1}) with eltype NamedTuple{(:a, :b),Tuple{Int64,String}}:
Error showing value of type StructArray{NamedTuple{(:a, :b),Tuple{Int64,String}},1,NamedTuple{(:a, :b),Tuple{Array{Int64,1},Array{String,1}}},Int64}:

(I could not find this case noted in other places - it seems for me to be a bug, but maybe it is intended for performance reasons?)

bkamins avatar Jun 07 '20 17:06 bkamins

Yes, right now if you push directly to a StructArray with the wrong type, the behavior is undefined.

There is a StructArrays.append!! method that does "mutate or expand" (along the lines of BangBang), and is used to collect iterables where the eltype may change. Runtime performance of StructArrays.append!!(v, (el,)) should, at least in principle, be comparable with push!(v, el).

It would require some benchmarking, but maybe it's possible to use that machinery to see at compile time whether push!, append! and setindex! would error, and then throw before messing around with the various columns.

piever avatar Jun 12 '20 17:06 piever