Query.jl

Query.jl copied to clipboard

Hi! There is a very confusing error thrown in some cases with mixed types.

A minimal example:

    df = DataFrame(a=1:1)
    df |>
        @mapmany([(c=Inf,), (c=1,)], (; _..., b=__)) |>
        @groupby({_.b}) |>
        @map((; key(_)..., a=_.a))

throws type NamedTuple has no field a on the last line with @map. This is due to c having type Float in one row, and Int in another. When all values have the same type, the same code works:

    df = DataFrame(a=1:1)
    df |>
        @mapmany([(c=0,), (c=1,)], (; _..., b=__)) |>  # changed Inf to 0
        @groupby({_.b}) |>
        @map((; key(_)..., a=_.a))

Julia 1.2.0, Query v0.12.1.

Yeah, the current design very much only works for "type stable iterators", i.e. iterators where each element has the same type. I'll have to think whether we can relax that.

For now a better error message would of course be helpful, but that might also not be easy...

Oh, didn't know that, and didn't see anywhere in the docs. Well, this is a pretty strong limitation to keep in mind... Especially as (to my understanding) it relies not only on all elements belonging to the same concrete type, but also the container having concrete eltype or analog - which is not the same. I think it depends on compiler type inference at least in some cases.