dask-sql icon indicating copy to clipboard operation
dask-sql copied to clipboard

[ENH] Add ~isna() support for predicate pushdown

Open ayushdg opened this issue 2 years ago • 0 comments

Is your feature request related to a problem? Please describe. A common filter applied to many sql queries is filtering out nulls for certain tables that usually get's pushed down to the TableScan step. We implement is not null as a combination of df.isna() chained with a not operation. It would be good to support identifying these patterns in the hlg for predicate pushdown.

Describe the solution you'd like

Describe alternatives you've considered

Additional context

ayushdg avatar Dec 15 '22 18:12 ayushdg