quickwit icon indicating copy to clipboard operation
quickwit copied to clipboard

Optimize simple time ranged search queries

Open tontinton opened this issue 7 months ago • 2 comments

When the search request contains a time range, we aborted the optimization of converting unneeded split searches into count queries.

Removed the TODO that was in the code for this.

Split the PR into 2: https://github.com/quickwit-oss/quickwit/pull/5758.

tontinton avatar Apr 19 '25 17:04 tontinton

Thanks for your PR. The way the logic is split here between optimize_split_order and optimize makes it very hard to proof read. Currently it's already a bit confusing because the optimize() logic for each variant depends on the sort order which is different for each variant. It would be good if we could avoid tightening the coupling between the two match statements even further by making the sort orders more complex.

It is a bit confusing, I'll try to think of a better way to do this when I have some more time.

tontinton avatar Apr 22 '25 20:04 tontinton

I would try refactoring this into:

fn optimize()
  match self {
    CandSplitDoBetter::SplitIdHigher(_) => optimize_split_id_higher()
    CandSplitDoBetter::SplitTimestampLower(_) => optimize_split_timesamp_lower()
    ...
    CanSplitDoBetter::Uninformative => {}
  }

where optimize_split_xxx:

  • sorts
  • returns early if !is_simple_all_query
  • applies its variant specific optimization

This would regroup the optimizations logics with the sorts they depend on to be correct. (To help the review, ideally, re-implement the current logic in 1 commit, and in a separate commit add you logic extension.)

Thanks!

rdettai avatar Apr 23 '25 10:04 rdettai