afayngelerindbx

I've run into a similar issue running a stress test on Loki. After digging around in the flusher code, I have some thoughts: 1. It seems that flush_op_timeout limits the...

X-posting here in case it helps folks: I have gotten extra evidence that the issue(at least my issue) is sequential flushes. I added ``` "-store.chunks-cache.cache-stubs=true", "-store.chunks-cache.cache.enable-fifocache=true", ``` to the ingester...

Ran this patch in our prod environment. Here are some observations: Before prefetching indexes we were seeing compactions take 8-10m. Empty compactions were running in single digit seconds. With the...

Hi @sandeepsukhani! Following up on our slack thread, I've changed the patch to parallelize fetching only the tenants that appear in the uncompacted indexes. The reason I chose this approach...

Initial internal numbers have been fairly successful with this patch. Compactions went from ~5-6m to ~2-3m. Since I posted the previous graph, we added a good number of streams to...

Hi @sandeepsukhani, this diff has been running in prod for us without issues for ~2 weeks. The perf gains were sustained. Please, let me know whether you have any feedback...

Release pipeline passed after rebase but not the `loki-mixin-check` not sure what to make of that.

Haven't had time to follow up on this. Not sure it is still necessary.