Cloudwatch Moto implementation does not support Expressions

[Stukongeluk]

Reporting Bug

Moto version: 1.3.16 Python version: 3.8 Boto3 version: 1.14.63

Installed this in my venv with pip install moto

How to reproduce

Use the cloudwatch client with get_metric_data with an expression.

def test_function():
 result = cloudwatch.get_metric_data(
        MetricDataQueries=[
            {
                'Id': 'expression',
                'Expression': '(totalBytes/1048576)/PERIOD(totalBytes)*100,
                'Label': 'e1',
                'ReturnData': True,
            },
            {
                'Id': 'totalBytes',
                'MetricStat': {
                    'Metric': {
                        'Namespace': 'AWS/EFS',
                        'MetricName': 'TotalIOBytes',
                        'Dimensions': [
                            {
                                'Name': 'FileSystemId',
                                'Value': 'somestring'
                            },
                        ]
                    },
                    'Period': 60,
                    'Stat': 'Sum',
                    'Unit': 'Bytes',
                },
                'ReturnData': False,
            },
          ],
        StartTime=current_datetime - timedelta(minutes=5),
        EndTime=current_datetime,
        ScanBy='TimestampDescending',
        MaxDatapoints=123
      )

An example stacktrace when testing this with Pytest and a test setup looking something like this:

import boto3
from moto import mock_cloudwatch
import pytest

@pytest.fixture(scope='function')
def aws_credentials():
    """Mocked AWS Credentials for moto."""
    os.environ['AWS_ACCESS_KEY_ID'] = 'testing'
    os.environ['AWS_SECRET_ACCESS_KEY'] = 'testing'
    os.environ['AWS_SECURITY_TOKEN'] = 'testing'
    os.environ['AWS_SESSION_TOKEN'] = 'testing'

@pytest.fixture(scope='function')
def cloudwatch(aws_credentials):
    with mock_cloudwatch():
        yield boto3.client('cloudwatch', region_name='eu-west-1')

@mock_cloudwatch
def test__something():
from cloudwatch_example import test_function

boto3.client('cloudwatch', region_name='eu-west-1')
result = test_function()

This results in something like this:

self = <moto.cloudwatch.models.CloudWatchBackend object at 0x000001ACF01AED00>
queries = [{'expression': '(totalBytes/1048576)/PERIOD(totalBytes)*100/(throughput/1048576)', 'id': 'calculatedThroughput', 'lab..._stat._metric._dimensions.member.1._value': 'test_id', 'metric_stat._metric._metric_name': 'PermittedThroughput', ...}]
start_time = datetime.datetime(2020, 9, 18, 13, 32, 49, 853054, tzinfo=tzutc()), end_time = datetime.datetime(2020, 9, 18, 13, 37, 49, 853054, tzinfo=tzutc())     

    def get_metric_data(self, queries, start_time, end_time):
        period_data = [
            md for md in self.metric_data if start_time <= md.timestamp <= end_time
        ]
        results = []
        for query in queries:
>           query_ns = query["metric_stat._metric._namespace"]
E           KeyError: 'metric_stat._metric._namespace'

venv\lib\site-packages\moto\cloudwatch\models.py:339: KeyError

Expected

I expected that it wouldn't return an error, because this key is not required when trying to query an expression. In other words, I expected a MetricDataResults dictionary returned. Refering to: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/cloudwatch.html#CloudWatch.Client.get_metric_data It seems like the implementation for the CloudWatchBackend always requires a MetricStat with a Metric including a Namespace.

[bblommers]

Correct @Stukongeluk - the get_metric_data-implementation is quite bare-bone at the moment. Marking this as an enhancement.

Thanks for raising it!