cortex-tools icon indicating copy to clipboard operation
cortex-tools copied to clipboard

Add function to analyse used labels, and not just used metrics.

Open tomwilkie opened this issue 3 years ago • 1 comments

This is a WIP PR which analyses the usage of labels, and not just metric names. The idea is eventually we'll be able to identify labels we can drop / aggregate way.

Lots of this exists already (great work @gouthamve) but AFAICT only analyses metric names? I plan on just working through this pipeline and updating everything to consider labels too.

 1.Extract Queries     2.Line-delimited  3.Extract Labels
                         PromQL queries
┌────────────────────┐
│  Prometheus Logs   ├─────────┐
└────────────────────┘         │
                               │
┌────────────────────┐         │         ┌───────────────┐
│  Prometheus Rules  ├─────────┼────────►│ Query Analyse │
└────────────────────┘         │         └───────┬───────┘
                               │                 │
┌────────────────────┐         │                 │
│ Grafana Dashboards ├─────────┘                 │
└────────────────────┘                           │
                                                 │
                                                 │
                                                 │
 4.Extract Series Labels                         │   5. Find unused labels
                                                 ▼
┌────────────────────┐                   ┌───────────────┐
│    Active Series   ├──────────────────►│   Label Diff  │
└────────────────────┘                   └───────┬───────┘
                                                 │
                                                 │
                                                 │
                                                 │
                                                 │
 6.Extract Series Counts                         │   7. Rank unused labels
                                                 ▼      to find best
┌────────────────────┐                   ┌───────────────┐
│    Active Series   ├──────────────────►│  Rank Unused  │
└────────────────────┘                   └───────┬───────┘
                                                 │
                                                 │
                                                 ▼
                                         ┌───────────────┐
                                         │  Aggregation  │
                                         │     Rules     │
                                         └───────────────┘

Signed-off-by: Tom Wilkie [email protected]

tomwilkie avatar Jan 06 '22 14:01 tomwilkie

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Jun 15 '22 17:06 CLAassistant