iceberg icon indicating copy to clipboard operation
iceberg copied to clipboard

API: Binder should not simplify isNull and notNull when their reference …

Open zhongyujiang opened this issue 2 years ago • 3 comments

…fields are required but nested in optional parents.

Currently, Binder will simpify isNull and notNull to alwaysFalse and alwaysTrue respectively, when the expressions are to be bound to required fields. However a required field can still be null in the row when it is nested in an optional parent field. Therefore, this causes data to be incorrectly skipped when querying for rows where their nested field(required sub field nested in optional parent) is null.

This changes the Binder to only simplify nullability predicates when their target field and all its parents are all required, to fix this issue.

zhongyujiang avatar Apr 14 '23 13:04 zhongyujiang

Seems like a good catch. I'll try to review the actual fix later this week unless someone gets there earlier.

cc @nastra @rdblue @szehon-ho @flyrain @amogh-jahagirdar @jackye1995

aokolnychyi avatar Apr 18 '23 06:04 aokolnychyi

Hey @aokolnychyi , do you have time to review this?

zhongyujiang avatar May 04 '23 11:05 zhongyujiang

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

github-actions[bot] avatar Aug 29 '24 00:08 github-actions[bot]

This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

github-actions[bot] avatar Sep 05 '24 00:09 github-actions[bot]