dask-sql icon indicating copy to clipboard operation
dask-sql copied to clipboard

More restrictive input validation during predicate pushdown filter extraction

Open charlesbluca opened this issue 2 years ago • 1 comments

While running some TPC-H queries using dask-sql + dask-cudf, I noticed some failures around predicate pushdown that seemed to be caused by us incorrectly pushing down blockwise comparisons between columns.

This PR attempts to tighten the logic of _blockwise_comparison_dnf to block this edge case; marking this as draft as I'm not sure if we want to block all RHS getitem calls yet.

cc @rjzamora

charlesbluca avatar Nov 20 '23 16:11 charlesbluca

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (2f068f9) 85.55% compared to head (d841dcc) 85.51%.

Files Patch % Lines
dask_sql/physical/utils/filter.py 0.00% 1 Missing and 1 partial :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1269      +/-   ##
==========================================
- Coverage   85.55%   85.51%   -0.05%     
==========================================
  Files          77       77              
  Lines        4257     4259       +2     
  Branches      758      759       +1     
==========================================
  Hits         3642     3642              
- Misses        446      447       +1     
- Partials      169      170       +1     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Nov 20 '23 19:11 codecov-commenter