pyroscope icon indicating copy to clipboard operation
pyroscope copied to clipboard

WIP: Support avg time_series aggregation

Open simonswine opened this issue 5 months ago • 0 comments

This implements an avg time series aggregation. Achieving an avg aggration is not trivial:

  • On ingest we split each profile series into smaller smaller ones, by using the dynamic labels.

This PR will reaassemble profiles matching with matching ID/timestamp and build an avg per series and the aggregate the averages per sum.

So lets say we have a 3 replica deployment with each using exactly 1GiB of in use memory.

We would be expect each pods series to be a flat 1GiB (no matter how many points fall into the step size) and when those 3 pods are combined, we would expect a flat 3GiB line.

Drawing:


                                                       │                                                                                    
                     Step A: 0 - 30s                   │         Step B: 30 - 60s                                                           
                                                       │                                                                                    
──────────────x───────────────────────────x────────────┼──────────────────x───────────────────────────x──        Pod A (scape interval 15s) 
              2 GiB                       1.8 GiB      │                  2.4 GiB                     2 GiB                                 
                                                       │                                                                                    
                                                       │                                                                                    
                                                       │                                                                                    
                                                       │                                                                                    
─x──────────────────────────x──────────────────────────┼───x──────────────────────────────x──────────────        Pod B (scrape interval 15s)
 32GiB                      40GiB                      │   41 GiB                         42 GiB                                            
                                                       │                                                                                    
                                                       │                                                                                    
                                                       │                                                                                    
─────────x─────────────────────────────────────────────┼──────────x──────────────────────────────────────        Pod C (scrape interval 30s)
         0.5GiB                                        │          0.5GiB                                                                    
                                                       │                                                                                    
                                                       │                                                                                    
                                                       │                                                                                    
      Avg aggreation:                                  │                                                                                    
                                                       │                                                                                    
       Pod A :   1.9 GiB                               │     Pod A :   2.2 GiB                                                              
       Pod B :  36.0 GiB                               │     Pod B :  41.5 GiB                                                              
       Pod C :   0.5 GiB                               │     Pod C :   0.5 GiB                                                              
       -----------------                               │     ----------------                                                               
                38.4 GiB                               │                                                                                    

simonswine avatar Jul 14 '25 13:07 simonswine