Bug: `addDimensions()` should create a set

[dreamorosi]

Expected Behavior

When calling the metrics.addDimensions() method of the Metrics utility, the dimensions should create a new dimension set rather than concatenating the dimensions into the existing set.

Basically this:

const metrics = new Metrics({
  namespace: 'serverlessAirline',
  serviceName: 'orders',
});

export const handler = async () => {
  metrics.addDimension('environment', 'prod');

  metrics.addDimensions({
    dimension1: "1",
    dimension2: "2"
  });

  // continue emitting & flushing metrics as today
  metrics.addMetric('foo', MetricUnit.Count, 1);
  metrics.publishStoredMetrics();
};

should generate this:

{
   "_aws":{
      "CloudWatchMetrics":[
         {
            "Namespace":"A",
            "Dimensions":[
               ["dimension1"],
               ["dimension1", "dimension2"],
            ],
            "Metrics":[
               {
                  "Name":"foo",
                  "Unit":"Bytes"
               }
            ]
         }
      ]
   },
   "dimension1":"1",
   "dimension2":"2",
   "foo":[1.0]
}

Current Behavior

Currently this:

const metrics = new Metrics({
  namespace: 'serverlessAirline',
  serviceName: 'orders',
});

export const handler = async () => {
  metrics.addDimension('environment', 'prod');

  metrics.addDimensions({
    dimension1: "1",
    dimension2: "2"
  });

  // continue emitting & flushing metrics as today
  metrics.addMetric('foo', MetricUnit.Count, 1);
  metrics.publishStoredMetrics();
};

instead generates this:

{
   "_aws":{
      "CloudWatchMetrics":[
         {
            "Namespace":"A",
            "Dimensions":[
               ["dimension1", "dimension2"],
            ],
            "Metrics":[
               {
                  "Name":"foo",
                  "Unit":"Bytes"
               }
            ]
         }
      ]
   },
   "dimension1":"1",
   "dimension2":"2",
   "foo":[1.0]
}

Code snippet

See above.

Steps to Reproduce

See above.

Possible Solution

The EMF specification already supports this type of use case, and specifically, it represents a dimension set as an array/list of dimensions within the Dimensions array/list of a metric, for example:

{
   "_aws":{
      "CloudWatchMetrics":[
         {
            "Namespace":"A",
            "Dimensions":[
               ["dimension1"],
               ["dimension1", "dimension2"],
               ["dimension1", "dimension2", "dimension3"]
            ],
            "Metrics":[
               {
                  "Name":"foo",
                  "Unit":"Bytes"
               }
            ]
         }
      ]
   },
   "dimension1":"1",
   "dimension2":"2",
   "dimension3":"3",
   "foo":[1.0]
}

In terms of developer experience, this is how the updated addDimensions() would work alongside existing functions:

Adding a new dimension set:

const metrics = new Metrics({
  namespace: 'serverlessAirline',
  serviceName: 'orders',
});

export const handler = async () => {
  // existing functionality for single dimension
  metrics.addDimension('environment', 'prod');

  // fixed feature
  metrics.addDimensions({
    dimension1: "1",
    dimension2: "2"
  });

  // continue emitting & flushing metrics as today
  metrics.addMetric('successfulBooking', MetricUnit.Count, 1);
  metrics.publishStoredMetrics();
};

Handling overwritten dimensions

const metrics = new Metrics({
  namespace: 'serverlessAirline',
  serviceName: 'orders',
});

export const handler = async () => {
  metrics.addDimension('dimension1', 'A');

  metrics.addDimensions({
    dimension1: "B", // last value to be set is used
    dimension2: "2"
  });
};

Click to see EMF output

{
   "_aws":{
      "CloudWatchMetrics":[
         {
            "Namespace":"A",
            "Dimensions":[
               ["dimension1"],
               ["dimension1", "dimension2"]
            ],
            "Metrics":[
               {
                  "Name":"foo",
                  "Unit":"Bytes"
               }
            ]
         }
      ]
   },
   "dimension1":"B",
   "dimension2":"2",
   "foo": [1.0]
}

Interaction with default dimensions:

const metrics = new Metrics({
  namespace: 'serverlessAirline',
  serviceName: 'orders',
  defaultDimensions: {
    environment: 'prod'
  }
});

export const handler = async () => {
  metrics.addDimensions({ // this set will include `environment` from default dimensions
    dimension1: "1",
    dimension2: "2"
  });
};

Click to see EMF output

{
   "_aws":{
      "CloudWatchMetrics":[
         {
            "Namespace":"A",
            "Dimensions":[
               ["environment", "dimension1", "dimension2"]
            ],
            "Metrics":[
               {
                  "Name":"foo",
                  "Unit":"Bytes"
               }
            ]
         }
      ]
   },
   "environment":"prod"
   "dimension1":"1",
   "dimension2":"2",
   "foo": [1.0]
}

Note: when default dimensions are added at runtime, for example via the metrics.setDefaultDimensions() method, only dimension sets added after the default dimensions were set will include these dimensions. Changes are not retroactive.

const metrics = new Metrics({
  namespace: 'serverlessAirline',
  serviceName: 'orders',
  defaultDimensions: {
    environment: 'prod'
  }
});

export const handler = async () => {
  metrics.addDimensions({"dimension1": "1", "dimension2": "2"}) # this includes environment

  // for some reason I want to add more default dimensions
  metrics.setDefaultDimensions({ tenantId: "1"}) // this does not set tenant_id retroactively into the previous set
  metrics.addDimensionSet({"foo": "1", "bar": "2"}) # this includes environment and tenantiId
}

---

In terms of implementation, below is a non-exhaustive list of actions that should be done:

• update implementation of addDimensions() here to create a set instead of multiple separate dimensions
• update documentation
• update unit tests under packages/metrics/tests/unit/dimensions.ts

Powertools for AWS Lambda (TypeScript) version

latest

AWS Lambda function runtime

22.x

Packaging format used

npm

Execution logs

[dreamorosi]

@mfigus can you please leave a comment here so I can assign the issue to you, since you're working on it.

[matteofigus]

There are a number of things here that seem not right. I based the PR on the assumption that these are mistakes on your side but perhaps we need to discuss. Here is a distinct list (many of these things are present in all examples):

1. Namespace not matching

namespace: 'serverlessAirline', ... "Namespace":"A",

I am assuming this is an issue on your side as I would expect the namespace on the output to match the one on the input.

2. Metric output format

metrics.addMetric('foo', MetricUnit.Count, 1); ... "foo":[1.0]

The current behaviour today, is that "foo" is 1 not [1.0]. If this needs to change, I think that's another issue independent from this Issue/PR.

3. Dimensions from addDimension should match Dimensions

metrics.addDimension('environment', 'prod');

metrics.addDimensions({ dimension1: "1", dimension2: "2" }); ... "Dimensions":[ ["dimension1", "dimension2"], ],

I believe this should actually be "Dimensions": [["environment"],["dimension1", "dimension2"]]?

4. addDimensionSet

metrics.addDimensionSet({"foo": "1", "bar": "2"}) # this includes environment and tenantiId

I am assuming you meant addDimensions here.

[dreamorosi]

Hi, sorry for the confusion. I copy/pasted these from different points of another conversation and some things got mixed up.

1. Correct, it should have been serverlessAirline like in the constructor.
2. The correct one is 1 - that output with [1.0] is from Python which deals with float numbers.
3. Yes, you're right.
4. Correct.

[powertools-for-aws-oss-automation[bot]]

[!warning] This issue is now closed. Please be mindful that future comments are hard for our team to see. If you need more assistance, please either reopen the issue, or open a new issue referencing this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

[github-actions[bot]]

This is now released under v2.24.0 version!