t-digest icon indicating copy to clipboard operation
t-digest copied to clipboard

Merge TDigest with different compression factors

Open jameswang2015 opened this issue 5 months ago • 3 comments

Sometimes we get a TDigest field with different compression factors(For example, we change compression factor but some users still use old versions of product that still have old compression factors). The current Merge function can't handle that. Can we extend the Merge function to support that?

On the other hand, we currently follow these steps to merge it. Would this make sense?

  1. Destruct the tdigest into a list of centroids and weights.
  2. Unnest the list of centroids and weights.
  3. Merge the centroids and weights into a single tdigest with the specified compression by using TDIGEST_AGG(m, w, compression)

Thanks.

jameswang2015 avatar Sep 24 '24 19:09 jameswang2015