mimir
mimir copied to clipboard
grafana
Expose thanos object storage option to download / upload in parallel
Is your feature request related to a problem? Please describe.
https://github.com/thanos-io/thanos/commit/79ab7c65cb4b66b9dcc4fa537cb43b00cc65066c
Thanos added the ability to download and upload blocks in parallel. This seems like a useful setting to make available (or set to some default value other than 1).
Describe the solution you'd like
Let this setting be configured to speed up compactor download and upload of TSDB blocks.
Describe alternatives you've considered
Nothing so far.
Is this related? #552
Yep, related but not exactly the same AFAICT. #552 downloads multiple blocks in parallel while the change referenced here would download each file that is part of a block in parallel. Given that, maybe it doesn't make sense to expose this setting. A benchmark would be required to tell if there's any benefit to doing this.
I would like to first learn which Mimir component would benefit from it.
The compactor is currently the only component that calls the Thanos block.Download() method which is where this would apply.
I would like to first learn which Mimir component would benefit from it.
The compactor is currently the only component that calls the Thanos
block.Download()method which is where this would apply.
Mimir download multiple source blocks in parallel. Having a double parallelism could be harder to reason about in terms of max concurrency.