Bugfix: prevent certain `not (x and y)` filters from causing a NPE

[itschrispeck]

We've seen some NPEs caused by certain combinations of filters, namely when the filtering follows the pattern: WHERE NOT (<scan filter> AND <filter on transform function>), and the docIds returned by the <filter on transform function> are less than the doc Ids from the proceeding <scan filter>.

example stack trace:

java.lang.NullPointerException
	at org.apache.pinot.core.operator.dociditerators.ExpressionScanDocIdIterator.advance(ExpressionScanDocIdIterator.java:111)
	at org.apache.pinot.core.operator.dociditerators.AndDocIdIterator.next(AndDocIdIterator.java:51)
	at org.apache.pinot.core.operator.dociditerators.NotDocIdIterator.next(NotDocIdIterator.java:48)
	at org.apache.pinot.core.operator.DocIdSetOperator.getNextBlock(DocIdSetOperator.java:75)
	at org.apache.pinot.core.operator.DocIdSetOperator.getNextBlock(DocIdSetOperator.java:39)

I'm not totally sure this is the best way to fix it - the bug is easily reproducible w/ the added unit test testTransformDocIdsLessThanFiltered.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 62.08%. Comparing base (59551e4) to head (f0e3a8a). Report is 715 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #13527      +/-   ##
============================================
+ Coverage     61.75%   62.08%   +0.33%     
+ Complexity      207      198       -9     
============================================
  Files          2436     2557     +121     
  Lines        133233   140889    +7656     
  Branches      20636    21856    +1220     
============================================
+ Hits          82274    87476    +5202     
- Misses        44911    46794    +1883     
- Partials       6048     6619     +571     

Flag	Coverage Δ
custom-integration1	<0.01% <0.00%> (-0.01%)	:arrow_down:
integration	<0.01% <0.00%> (-0.01%)	:arrow_down:
integration1	<0.01% <0.00%> (-0.01%)	:arrow_down:
integration2	0.00% <0.00%> (ø)
java-11	62.03% <100.00%> (+0.32%)	:arrow_up:
java-21	61.97% <100.00%> (+0.35%)	:arrow_up:
skip-bytebuffers-false	62.05% <100.00%> (+0.30%)	:arrow_up:
skip-bytebuffers-true	61.95% <100.00%> (+34.22%)	:arrow_up:
temurin	62.08% <100.00%> (+0.33%)	:arrow_up:
unittests	62.08% <100.00%> (+0.33%)	:arrow_up:
unittests1	46.67% <100.00%> (-0.22%)	:arrow_down:
unittests2	27.64% <0.00%> (-0.09%)	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[itschrispeck]

Good catch! The root cause is that NotDocIdIterator breaks the contract that no more calls to next() or advance() are allowed after they return EOF. We should move modify NotDocIdIterator.next() to perform the check of if (_nextDocId >= _numDocs) first

I see it now, thanks! I think the original check is still valid, I duplicated it to also check first