thanos icon indicating copy to clipboard operation
thanos copied to clipboard

proxy: pre-calculate hash at leaf nodes

Open GiedriusS opened this issue 2 years ago • 4 comments

Is your proposal related to a problem?

We could increase the performance by calculating the hash of a chunk at the leaf node instead of the querier: https://github.com/thanos-io/thanos/pull/5296#discussion_r966899425

Describe the solution you'd like

Add another field with the hash and use it.

GiedriusS avatar Sep 09 '22 10:09 GiedriusS

Add another field with the hash and use it.

Can you please elaborate more on where to add the field for hashes? The Series API?

yeya24 avatar Sep 09 '22 16:09 yeya24

Once we have more details here, I would be inclined to take a stab at solving this one.

pedro-stanaka avatar Sep 14 '22 08:09 pedro-stanaka

Idea is to calculate this hash by the sender so that it would be less work on the querier (:

GiedriusS avatar Sep 14 '22 09:09 GiedriusS

https://github.com/thanos-io/thanos/blob/32a6c4e03a/pkg/store/storepb/types.proto#L23 If I understand correctly, we can add one more nullable hash field to the chunk itself and populate it at stores?

yeya24 avatar Sep 14 '22 18:09 yeya24

@GiedriusS can we close this in light of #5703 ?

pedro-stanaka avatar Nov 10 '22 10:11 pedro-stanaka

Yep :+1: thanks for your work on this! :muscle:

GiedriusS avatar Nov 10 '22 10:11 GiedriusS