Feature Request: Support Metric Math

[sidewinder12s]

This appears to be blocked by #134 & using the GetMetricData API.

https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/using-metric-math.html#metric-math-syntax

Some metrics are only available by computing them using Metric Math, like the actual value of Service Quotas.

(usage_data/SERVICE_QUOTA(usage_data))*100

The other popular cloudwatch_exporter has a similar feature request: https://github.com/nerdswords/yet-another-cloudwatch-exporter/issues/138

[matthiasr]

Thank you, I think that's a valid request, but it is a ways off as you noted. Given YACE is closer to having it, I would however encourage implementing it over there first.

[or-shachar]

@sidewinder12s @matthiasr - now that we support GetMetricData - we might be just as close as YACE to metric math.

Our company doesn't use metric math so I'm not sure it's a priority for us to implement it but it might be worth talking about how we would want this feature to be used.

@sidewinder12s - given our current configuration model - how would you change it to support metric math. IIUC - it'll need to support (ordered?) set of named expressions and a final expression. How would you accommodate it in the CloudwatchExporter configuration?

(It'll be very helpful if you could compose a desired example configuration).

[sidewinder12s]

Here is an example of what the Service Quota expression queries look like in CloudWatch (from the source view).

{
    "metrics": [
        [ "AWS/Usage", "ResourceCount", "Class", "Standard/Spot", "Resource", "vCPU", "Service", "EC2", "Type", "Resource", { "id": "usage_data", "color": "#1f77b4", "stat": "Sum", "period": 300, "visible": false } ],
        [ { "expression": "(usage_data/SERVICE_QUOTA(usage_data))*100", "label": "% Utilization", "id": "pct_utilization", "color": "#1f77b4", "region": "us-east-1", "period": 60 } ]
    ],
    "view": "timeSeries",
    "region": "us-east-1",
    "title": "All Standard (A, C, D, H, I, M, R, T, Z) Spot Instance Requests - (Utilization %)",
    "stat": "Maximum",
    "period": 60
}

I am not super familiar with how complicated these kinds of queries can get. My nieve thought would be adding a new sub-dict that lets you have a primary query and then an expression query.

- aws_namespace: AWS/Usage
  aws_metric_name: ResourceCount
  aws_dimensions: [Service, Resource, Type, Class]
  aws_dimension_select: 
    Service: EC2
    Resource: vCPU
    Type: Resource
    Class: Standard/Spot
  period_seconds: 300
  aws_expression:
    query: (queried_metric/SERVICE_QUOTA(queried_metric))*100
    period_seconds: 300

queried_metric might just need to be canned or maybe add another top level key of id similar to how the console does it.

[matthiasr]

Looking at the API, it seems that you can either ask for a MetricStat or an expression in each individual query, but you can reference (named) metrics that you are requesting at the same time. This suggests to me that we should not allow mixing aws_metric_name and aws_expression in one mapping.

Additionally, how would I express in configuration if I want to combine two metrics in my expression, such as this example?