dask-sql
dask-sql copied to clipboard
[ENH] Add ~isna() support for predicate pushdown
Is your feature request related to a problem? Please describe.
A common filter applied to many sql queries is filtering out nulls for certain tables that usually get's pushed down to the TableScan step. We implement is not null as a combination of df.isna() chained with a not operation. It would be good to support identifying these patterns in the hlg for predicate pushdown.
Describe the solution you'd like
Describe alternatives you've considered
Additional context