Christopher Peck

Improved segment build time for Lucene text index realtime to offline conversion

1

**Problem:** We typically see long (7-10min) segment build times when using Lucene index with 1-1.5GB segment sizes. 70-80% of this time is spent building the Lucene text index. **Background:** In...

SelectContext does not cancel when context is timed out

4

I'm using the SelectContext method, and trying to timeout the context to cancel indefinitely running queries. It appears that the timeout is not respected, and the goroutine hangs forever. It...

Improve realtime Lucene text index freshness/cpu/disk io usage

3

This PR allows for better freshness/cpu/disk io usage for realtime Lucene text index. User facing changes: 1. Add config `pinot.server.lucene.min.refresh.interval.ms` (default is 10, as 10ms was the previous behavior) 2....

Fix bug when evaluating resource status during service startup check

1

We have seen servers sometimes fail to pass the service status checker until the timeout is reached, even after all segments are online/in the expected state. Logs show: ``` Sleep...

Bugfix: prevent certain `not (x and y)` filters from causing a NPE

2

We've seen some NPEs caused by certain combinations of filters, namely when the filtering follows the pattern: `WHERE NOT ( AND )`, and the docIds returned by the `` are...

Fix TextMatchFilterOptimizer `or not` behavior

1

Lucene's query language has a constraint where `NOT` operator cannot be used with just one term, since it relies on the difference of sets. Therefore, the pinot query `x OR...

### Problem Currently, Pinot's `RealtimeLuceneTextIndex` uses Lucene's near real-time indexing functionality. Some [effort](https://github.com/apache/pinot/pull/13503) has been made to reduce the delay already. However, due to the nature of the implementation true...

bugfix: TextMatchFilterOptimizer grouping for inner compound query

1

I missed this edge case when adding TextMatchFilterOptimizer. For a query such as `text_match(col1, 'aaa') AND text_match(col1, 'bbb OR ccc')`, this would have previously been optimized to `text_match(col1, '(foo AND...