lumigo-node icon indicating copy to clipboard operation
lumigo-node copied to clipboard

Execution tags globals list not cleared unless tracing

Open benheymink opened this issue 3 years ago • 7 comments
trafficstars

Describe the bug I've followed the docs here to add execution tags to our functions. Note that we are not currently tracing these functions, just monitoring their invocations with Lumigo.

It looks like when a busy lambda function is kept alive, the list of execution tags are stored in a global variable that will continue to grow and never be cleared (clearing this list appears to only be performed at the end of tracing, which again - we're not using)

To Reproduce Steps to reproduce the behavior:

  1. Enable Lumigo integration with your AWS environment (don't enable tracing at this stage)
  2. Follow the docs here to add the node-tracer lib to your lambda function
  3. On invocation, attempt to write an execution tag with lumigo.addExecutionTag('<key>', '<value>');
  4. Repeatedly invoke this function, and once the list of tags has grown to 50, you will see the following message:.

INFO Lumigo Warning: Skipping addExecutionTag: Unable to add tag: maximum number of tags is 50: <key> - <value>

Expected behavior

  • Execution tags should be cleared at the end of each invocation, or a utility function should be exposed to the caller to perform manual clear up (i.e expose clearGlobals)
  • ExecutionTags should be useable without tracing

Runtime details

  • node v16

Screenshots N/A

Additional context N/A

benheymink avatar Aug 08 '22 08:08 benheymink

(Alternatively perhaps the trace wrapper for functions could include some sort of option as to if tracing should be enabled or not?)

benheymink avatar Aug 08 '22 08:08 benheymink

Hey @benheymink, thank you for the detailed report. Just to make sure I understand the scenario.

  • The Lambda is not traced
  • When using addExecutionTag more than 50 times you get the above error.

efimk-lu avatar Aug 08 '22 08:08 efimk-lu

@efimk-lu Yes, all correct. But we are not manually calling addExecutionTag more than 50 times, this is simply a lambda function being repeatedly invoked (and due to the frequency of the invocations, I think AWS is keeping it warm).

Normally that global list of tags is cleared at the end of a trace (see clearGlobals call within the endTrace method of src/tracer/tracer.ts)

benheymink avatar Aug 08 '22 08:08 benheymink

@benheymink got it

efimk-lu avatar Aug 08 '22 08:08 efimk-lu

@benheymink , thinking out loud on the right solution. Would silently ignoring the issue, e.g. don't print anything to logs when the Lambda is not traced would be good enough? Exposing a public API is a liability that we prefer to avoid.

efimk-lu avatar Aug 08 '22 08:08 efimk-lu

Well, ideally I'd prefer it if execution tags worked without tracing, which is what I was expecting (nothing in the docs for execution tags mentions the need for tracing). I appreciate that requires the need for wrapping the function (so that we know when the function has finished!) - so perhaps the answer here is to be able to provide some options/configuration to the trace wrapper to indicate that we don't want "full" tracing, just executionTags and programmatic errors?

benheymink avatar Aug 08 '22 08:08 benheymink

OK, got it. thx for the feedback.

efimk-lu avatar Aug 08 '22 08:08 efimk-lu

Hi @benheymink, we believe to have fixed this issue with #401. Execution tags require tracing (it is, after all, a tracing API), but they will not incorrectly accumulate anymore. Can you please confirm that the issue is solved for you?

mmanciop avatar Dec 02 '22 12:12 mmanciop

@mmanciop Thanks! Will report back shortly, may take a week or so for our upgrade to go through, but will report soon!

benheymink avatar Dec 02 '22 16:12 benheymink

Please reopen the issue if it is not solved.

mmanciop avatar May 24 '23 07:05 mmanciop