quickwit
quickwit copied to clipboard
delay fast field type conversion to harvest phase
extract_typed_sort_value_opt is in the collect hot loop and very performance sensitive. Delay conversion from u64 to actual type to harvest phase.
The impact on the histogram query time seems a little odd, probably related to some inlining. The impact on other sort query times roughly what I'd expect.
A sad bit is that we compare things twice:
* the search_after part is acting as a first fence. * the head of the top k collector is also acting as a fence.In reality... there is only one fence of course. We could have had single set of sort values on the segment collector (
current_score_fence) or something like that, initialized it on the search_after parameter, and then just used the regular collect_top_k logic.What makes everything so difficult is the different treatment of ties ( the
split_search_after_orderthing).
True, I think we could maybe have specialized SegmentCollectors with Box<SegmentCollector>, to provide specialized implementations via generics for the different use cases. The calling performance overhead in combination with collect_block should be fine.
should we merge this PR and open a follow-up issue then?
should we merge this PR and open a follow-up issue then?
Let's fix the bug and then merge. People do rely on edge.