Support AWS Lambda

[chemalopezp]

Description

This is the meta issue for AWS Lambda support for different agents (starting with NodeJS, Java and Python). The task includes:

• Providing language specific instrumentation for AWS Lambda
• Providing a AWS Lambda layer (if required) for the agent artifacts
• Creating a documentation that describes the setup of the AWS Lambda support for the different agents

Prev Description

The current Elastic APM Node.js agent is not working in AWS Lambda. While it used to work on previous versions (<v2) the current releases are not compatible with it.

There seems to be some digging by the Elastic team, see https://github.com/elastic/apm-agent-nodejs/issues/827

Agents status

Agent spec: https://github.com/elastic/apm/blob/main/specs/agents/tracing-instrumentation-aws-lambda.md

• [x] https://github.com/elastic/apm-agent-java/issues/1911
• [x] https://github.com/elastic/apm-agent-nodejs/issues/2156
• [x] https://github.com/elastic/apm-agent-python/issues/428
• [ ] https://github.com/elastic/apm-agent-dotnet/issues/1764

[Minu2020]

We would like to have AWS Lambda-django in Elastic APM. We are able to push the CPU and memory metrics.. Also transactions are appearing in Kibana. However, APM is not showing any transactions or Errors.

[alex-fedotyev]

Thanks for raising this! We will include it into prioritization going forward, although it will unlikely going to be implemented in the short term.

[nugroho-exp]

AWS has introduced Lambda extensions:

https://aws.amazon.com/blogs/compute/introducing-aws-lambda-extensions-in-preview/

It seems it is the way to go as other vendors like Datadog, NewRelic and AppDynamics have already released extensions for it.

[basepi]

Thanks for the link, @nugroho-expereo! We also saw this release and this is definitely the route we intend to take with our support.

[babyhuey]

Is there any type of idea when this is going to be looked at? We really would love to use this over xray/datadog in our Lambdas.

[MyEVerse]

@AlexanderWert this issue has been open quite a while, and I'm a little bummed to see it being pushed to the next release. We're clearly not the only ones using node.js Lambda functions on our back end and would really love to see this capability revived...Some Redditors are trying to use rum-js instead of this agent, in fact. It's been my desire to use the elastic cloud as a one-stop repository for logs and metrics, but we have to make architectural choices in real-time. Let us know if you have solid intel about how or when this is going to be a reality. Thanks.

[AlexanderWert]

@MyEVerse Thanks for the feedback!

Actually, we are actively working on this feature:

• https://github.com/elastic/apm-agent-nodejs/pull/2194
• https://github.com/elastic/apm-agent-python/pull/1193
• https://github.com/elastic/apm-agent-java/pull/1951

We cannot state any release dates but it won't take long anymore.

[nick-gray]

Potential alternative until the agents handle it https://www.elastic.co/blog/tracing-aws-lambdas-with-opentelemetry-and-elastic-observability

[AlexanderWert]

@MyEVerse and others: We have a preview version for AWS Lambda APM data support for Node.js and Python available. Here are the links to the documentation to set up things:

• Setting up the Lambda Extension
• AWS Lambda with the Node.js APM Agent --> use Release 3.24.0 of the agent
• AWS Lambda with the Python APM Agent --> use Release 6.6.1 of the agent

Happy to get some feedback to further improve our AWS lambda support.

[ankurthuse]

The Setting up Lambda Extension link is broken - I think the URL is supposed to be this.

https://github.com/elastic/apm-aws-lambda/tree/main/apm-lambda-extension

[trentm]

@ankurthuse Thanks. The "docs/" dir was moved up to the top of the repo, so currently that doc is at https://github.com/elastic/apm-aws-lambda/blob/main/docs/aws-lambda-extension.asciidoc

[AlexanderWert]

Done for Java, Node.js, Python

[AlexanderWert]

Reopened to track the status for other APM agents