powertools-lambda-python icon indicating copy to clipboard operation
powertools-lambda-python copied to clipboard

Allow log_metrics to be used as a context manager for testing subset functions

Open offbyone opened this issue 3 years ago • 2 comments

Use case

I am attempting to write a unit test for a piece of functionality that isn't in the top of the lambda function call stack, but is emitting metrics. Moreover, the functionality is an async coroutine, so it's not amenable to being wrapped directly with the @metrics.log_metrics() decorator.

What I want is to be able to do this:

@pytest.mark.asyncio
async def test_sub_functionality(capsys):
    metrics = Metrics(namespace="ThisTest")

    # this is the new bit:
    with metrics.log_metrics():
        await fixture.awaitable_thing()

    log = capsys.readouterr().out.strip()  # remove any extra line
    metrics_output = json.loads(log)  # deserialize JSON str
   assert "TheMetric" in metrics_output["_aws"]["CloudWatchMetrics"][0]["Metrics"][0]["Name"]

Solution/User Experience

Provide a context manager facade for the log_metrics decorator; either by factoring out the contents of that method into a context manager directly, or by implementing it in a different way.

Alternative solutions

Alternately, if `log_metrics()` worked with arbitrary functions (and coroutines) instead of being limited to applying to functions that match the Lambda function interface, I could probably manage using it in my tests more easily.

Acknowledgment

offbyone avatar May 26 '22 17:05 offbyone

Thanks for opening your first issue here! We'll come back to you as soon as we can.

boring-cyborg[bot] avatar May 26 '22 17:05 boring-cyborg[bot]

hey @offbyone thanks a lot for raising this to us :) I have a few questions

  • Are you primarily looking to confirm whether those metrics are being emitted? Or are they being flushed sooner than those coroutines hence the context manager?
    • e.g. assert "my_metric" in metrics.metric_set in the first case, as metric_set is our dict that holds metrics in memory before log_metrics flushed them at the end
  • Alternatively, would a synchronous method to serialize and flush metrics suffice here? Thinking of challenges we faced with colored functions, ctx manager, and generators before

Let's assume we go with a new context manager (need a new name), would you be open to contribute it after our pause? If we do hear from more customers that they're blocked, or we complete our E2E/Integ tests earlier, we can revisit this before end of July.

Look forward to hearing from you

Thanks!!

heitorlessa avatar May 30 '22 13:05 heitorlessa