quickwit icon indicating copy to clipboard operation
quickwit copied to clipboard

Published 20 hours ago • verified publisher icon quickwit-oss

Quickwit's indexing per-index footprint

Open fulmicoton opened this issue 2 years ago • 0 comments

Right now Quickwit uses several thread PER indexing pipeline. This ticket is about improving the situation. In the following N is hte number of indexes.

  • [ ] Avoid having N x k concurrent uploads by sharing the conccurrent upload semaphore,
  • [ ] Have configurable runtime
  • [ ] Add a no more message handler in the actor framwork
  • [ ] Modify the indexer actor to make sure we have at most K indexwriter at the same time.

fulmicoton avatar Jun 07 '22 05:06 fulmicoton