thanos-receive-controller icon indicating copy to clipboard operation
thanos-receive-controller copied to clipboard

Published 20 hours ago • verified publisher icon observatorium

main.go: enable cleanup of old PVCs

Open squat opened this issue 4 years ago • 4 comments

This commit enables the controller to clean up the PVCs of Thanos Receive Pods that are watched by it. Whenever a receiver is deleted, the controller will spawn a helper container that mounts all PVCs specified by the StatefulSet for that container, replicate the contents to object storage, and then rm -rf the contents.

Tested on a kind cluster. A follow up PR will add E2E tests and verify this functionality.

squat avatar Jan 17 '20 15:01 squat

This PR is currently blocked on https://github.com/observatorium/thanos-replicate/blob/eb8dd2444ac8a24b1cf2d458055d1e1c9f727591/scheme.go#L197.

Because there are no labels on the block before the receive shipper uploads them, thanos-replicate skips the block when running in the cleanup job.

Will make a PR to move this forward

squat avatar Jan 20 '20 16:01 squat

I also realized this won’t work anymore with multitsdb, we way want to implement that right away as I think the strategy might be a bit different.

brancz avatar Jan 21 '20 08:01 brancz

I also realized this won’t work anymore with multitsdb, we way want to implement that right away as I think the strategy might be a bit different.

Yeah, that sounds like a plan

squat avatar Jan 28 '20 17:01 squat