Query.jl
Query.jl copied to clipboard
Error when filtering by Union{Bool, Null}
On Julia v0.6, using DataFrames, DataValues and Query masters:
julia> using Query, DataFrames
julia> df = DataFrame(a=[1,2,3], b=Union{Null,Bool}[true, true, false], c=[true, true, false])
3×3 DataFrames.DataFrame
│ Row │ a │ b │ c │
├─────┼───┼───────┼───────┤
│ 1 │ 1 │ true │ true │
│ 2 │ 2 │ true │ true │
│ 3 │ 3 │ false │ false │
julia> df|>@where(_.c)|>DataFrame # 1. filtering by non-nullable, OK
2×3 DataFrames.DataFrame
│ Row │ a │ b │ c │
├─────┼───┼──────┼──────┤
│ 1 │ 1 │ true │ true │
│ 2 │ 2 │ true │ true │
julia> df|>@where(_.b)|>DataFrame # 2. filtering by nullable, FAILS
ERROR: TypeError: non-boolean (DataValues.DataValue{Bool}) used in boolean context
Stacktrace:
[1] start(::QueryOperators.EnumerableWhere{NamedTuples._NT_a_b_c{Int64,DataValues.DataValue{Bool},Bool},QueryOperators.EnumerableIterable{NamedTuples._NT_a_b_c{Int64,DataValues.DataValue{Bool},Bool},IterableTables.DataFrameIterator{NamedTuples._NT_a_b_c{Int64,DataValues.DataValue{Bool},Bool},Tuple{Array{Int64,1},Array{Union{Bool, Nulls.Null},1},Array{Bool,1}}}},##14#16}) at /home/astukalov/.julia/v0.6/QueryOperators/src/enumerable/enumerable_where.jl:33
[2] macro expansion at /home/astukalov/.julia/v0.6/IterableTables/src/integrations/dataframes-null.jl:91 [inlined]
....
julia> df|>@where(_.b==true)|>DataFrame # 3. comparison of nullable with the value, OK
2×3 DataFrames.DataFrame
│ Row │ a │ b │ c │
├─────┼───┼──────┼──────┤
│ 1 │ 1 │ true │ true │
│ 2 │ 2 │ true │ true │
I don't know how to properly fix the second test (I assume it should have the same behaviour as the 1st), e.g. should Base.convert(::Type{Bool}, ::DataValue{Bool}) be defined or filtering lambda generation tweaked?
Ah, I guess it would be fixed by davidanthoff/QueryOperators.jl#2
I think I'd also be fine with a solution where DataValue(true) and DataValue(false) would be accepted by @where, but an error would be thrown if a DataValue{Bool}() was returned. But let me think a bit more about it before I commit to it :)