trinity-1686a

we addad an index on splits(index_uid), but should still investigate other possible indexes

I've experimented with creating different indexes, with different level of usefulness: - an index on `split_state` doesn't appear to get used (probably too low selectivity) - BRIN on `time_range_start` and...

we've added some indexes already. We'll want to do something about tags at some point, but that requires making further changes, and the situation already improved a lot. Going to...

fixed by #4726

might be related to #4629

this sounds a lot like what routing expression are. We could add features to the dsl to allow this kind of things, and get the "new split on boundary crossed"...

you can use VRL to feed quickwit with csv, and transform it ```yaml # Your source config here # ... input_format: plain_text transform: script: | # csv looks like: "123;abc;def"...

I think at the bare minimum, we need to call `last_delete_opstamp()` for each index (to check it is zero), or in a future where `list_stale_splits()` supports None, one call to...

I've instrumented quickwit to know where allocation happened. On a quickwit running just the indexer service, with 1.5k indexes all getting fed lightly, i get 8.9GiB allocated memory with following...

with cooperative indexing, things are much more tame. If I increase the number of concurrent request (i'm currently at 800), memory is dominated by buffers allocated in warp/hyper, to handle...