Optimize simple time ranged search queries
When the search request contains a time range, we aborted the optimization of converting unneeded split searches into count queries.
Removed the TODO that was in the code for this.
Split the PR into 2: https://github.com/quickwit-oss/quickwit/pull/5758.
Thanks for your PR. The way the logic is split here between
optimize_split_orderandoptimizemakes it very hard to proof read. Currently it's already a bit confusing because theoptimize()logic for each variant depends on the sort order which is different for each variant. It would be good if we could avoid tightening the coupling between the twomatchstatements even further by making the sort orders more complex.
It is a bit confusing, I'll try to think of a better way to do this when I have some more time.
I would try refactoring this into:
fn optimize()
match self {
CandSplitDoBetter::SplitIdHigher(_) => optimize_split_id_higher()
CandSplitDoBetter::SplitTimestampLower(_) => optimize_split_timesamp_lower()
...
CanSplitDoBetter::Uninformative => {}
}
where optimize_split_xxx:
- sorts
- returns early if !is_simple_all_query
- applies its variant specific optimization
This would regroup the optimizations logics with the sorts they depend on to be correct. (To help the review, ideally, re-implement the current logic in 1 commit, and in a separate commit add you logic extension.)
Thanks!