thanos
thanos copied to clipboard
thanos-io
add proposal for parallel compaction
- [ ] I added CHANGELOG entry for this change.
- [X] Change is not relevant to the end user.
Changes
A proposal for scaling compactor via compacting of blocks for a single tenant in parallel. We have something similar in Cortex with https://cortexmetrics.io/docs/proposals/parallel-compaction/. I'm thinking that since Cortex uses Thanos under the hood for compaction, it would be beneficial for both projects if we can coordinate our effort on making compactor scale better, instead of having two projects work on their own solution.
I'm hoping that with this proposal, Cortex will be able to reuse the planner logic and the compaction logic. Both project should still implement how the coordination works, and in this PR, I've proposed something that might work for Thanos.
Verification
Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
![stale[bot] avatar](https://avatars.githubusercontent.com/in/1724?v=4 "Published by a pub.dev verified publisher"){kind=small}
Sorry for the delay. I hope we can continue this. Will do a another review next week.
Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.