Add series deletion with compactor proposal

[yeya24]

Signed-off-by: Ben Ye [email protected]

• [ ] I added CHANGELOG entry for this change.
• [x] Change is not relevant to the end user.

Changes

Add a proposal about series deletion. Continue work of https://thanos.io/tip/proposals-accepted/202012-deletions-object-storage.md/

For the reference implementation, please take a look at the two below:

1. Tombstone CLI and Store Gateway Series Masking
2. Tombstone Compactor

Verification

[stale[bot]]

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[matej-g]

Do we have any updates here @yeya24?

[yeya24]

Do we have any updates here @yeya24?

I am still looking for feedbacks from the community. I've already mentioned the approaches we could take so once we agree the proposal itself is good, we can merge and I'll open a pr. As listed in the PR description I already implemented one approach so feel free to take a look.

[stale[bot]]

Hello 👋 Looks like there was no activity on this amazing PR for the last 30 days. Do you mind updating us on the status? Is there anything we can help with? If you plan to still work on it, just comment on this PR or push a commit. Thanks! 🤗 If there will be no activity in the next week, this issue will be closed (we can always reopen a PR if you get back to this!). Alternatively, use remind command if you wish to be reminded at some point in future.

[stale[bot]]

Closing for now as promised, let us know if you need this to be reopened! 🤗

[stale[bot]]

Hello 👋 Looks like there was no activity on this amazing PR for the last 30 days. Do you mind updating us on the status? Is there anything we can help with? If you plan to still work on it, just comment on this PR or push a commit. Thanks! 🤗 If there will be no activity in the next week, this issue will be closed (we can always reopen a PR if you get back to this!). Alternatively, use remind command if you wish to be reminded at some point in future.

[paprickar]

Hey @yeya24,

I was looking for a solution to delete only specific time series (similar to what we can do with the thanos tool rewrite command "matcher").

I saw settings retention via compactor but this is globally only.

Your proposal would allow deletion of specific time series but only on demand if I understand it correct.

Therefore could we maybe extend the JSON to allow the "retention" use case?

maxAge: time in seconds how long you want to keep a metrics from Time.now() keep: if compactor should delete the tombstone file or not

{
        "ulid": "01G61GC0Q957TVZ2FC8S6Y2ZK7",
        "matchers": [
                "__name__=\"k8s_app_metric39\"",
                "cluster=\"us\"",
                "__block_id=\"01G61D0Q4BHTRC7HW08YRD0KGS\""
        ],
        "maxAge": 86400,
        "keep": true,
        "author": "Ben Ye",
        "reason": "not specified"
}

[stale[bot]]

Hello 👋 Looks like there was no activity on this amazing PR for the last 30 days. Do you mind updating us on the status? Is there anything we can help with? If you plan to still work on it, just comment on this PR or push a commit. Thanks! 🤗 If there will be no activity in the next week, this issue will be closed (we can always reopen a PR if you get back to this!). Alternatively, use remind command if you wish to be reminded at some point in future.

[paprickar]

@yeya24 @matej-g does one of you maybe have a status on this topic in general maybe?