quickwit icon indicating copy to clipboard operation
quickwit copied to clipboard

Add a gRPC batch leaf requests endpoint

Open fmassot opened this issue 2 years ago • 1 comments

In #3734, we added the multi indexes search.

The root search node now sends leaf requests for each index to leaf nodes. Consequently, the root node will need to merge more leaf responses. This can put a lot of pressure on the root node.

To avoid that, we should introduce a gRPC batch leaf request endpoint so the root node can send multiple leaf requests to a given leaf node. The leaf node will then execute the search on those leaf requests and pre-merge their results before sending the result back to the root.

fmassot avatar Aug 17 '23 16:08 fmassot

Using the pattern as suggested by @guilload here, sounds a good idea to solve this issue too.

fmassot avatar Aug 17 '23 16:08 fmassot

Closed via #4962.

guilload avatar Jun 18 '24 21:06 guilload