Query.jl icon indicating copy to clipboard operation
Query.jl copied to clipboard

Docs for missing values

Open baggepinnen opened this issue 4 years ago • 1 comments

I got confused by the following query

metrics = @from i in data0 begin
    @where i.happy != "NA" && i.energetic != "NA" && !ismissing(i.ActualStartTime)
    @select {i.ParticipantID, happy=round(Int, i.happy), energy=round(Int, i.energetic), i.sex, time=i.ActualStartTime}
    @collect DataFrame
end

and I expected the resulting dataframe to not contain any records for which time is missing, but no filtering took place. I had it explained for me on Zulip, that missing are translated to DataValues. The docs https://www.queryverse.org/Query.jl/stable/gettingstarted/#Missing-values-1 are not super clear on this and I did not expect this behavior even after having read that part of the docs again. In particular, the docs do not really mention what to use instead of ismissing or how I can opt out of the automatic translation if it all possible.

baggepinnen avatar May 15 '20 14:05 baggepinnen

This DataValue instead of missing business has confused me as well. Especially since I think there is no explanation in the documentation why the package is not using the standard Julia missing type.

Paethon avatar Jun 16 '20 09:06 Paethon