AWS SDK v3 Support (v2 deprecation)

[throrin19]

Problem Statement

Actually, @sentry/serverless package use the aws-sdk v2. It results on this warnings when testing in local with serverless-offline :

(node:49034) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.
Please migrate your code to use AWS SDK for JavaScript (v3).
For more information, check the migration guide at https://a.co/7PzMCcy

With lambdas using the node18 runtime, the sdk available is the v3, not the v2. To be able to use the sdk in v2, it must be included in the package, which makes the size of each lambdas quite substantial.

Solution Brainstorm

Use the sdk v3. But, with that, we have the problem of compatibility between aws node runtime < node18 using the sdk v2 and aws node runtime >= node18 using the sdk v3

[Lms24]

Hi @throrin19 thanks for writing in!

Sounds like we can't upgrade to the v3 SDK at the moment without dropping support for Node <18 on AWS Lambda. So we can't do this without releasing a major and even then it partially goes against our philosophy of supporting old runtimes as long as possible. On the other hand, I get that this warning is concerning. I'm going to backlog this issue and add it to the "Up for Discussion" tasks for our next major.

[throrin19]

Thanks for your response and I fully understand this complex problem

[alfaproject]

You don't need node 18. We are using the new SDK with node 16 and I'm pretty sure it works in node 14 and possibly even earlier versions. There's nothing special about their code

P.S.: anyone that is using the old SDK is already receiving this:

(node:28720) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. Please migrate your code to use AWS SDK for JavaScript (v3). For more information, check the migration guide at https://a.co/7PzMCcy

[Lms24]

According to the v3 docs, Node 14 is the minimum. But according to @throrin19,

we have the problem of compatibility between aws node runtime < node18 using the sdk v2 and aws node runtime >= node18 using the sdk v3

Does this mean that v3 can only be used in Node 18 lambdas? I'm a bit confused by this and would appreciate some more insights. Furthermore, I'm not sure (and didn't find yet) what the minimum version of v2 was.

Our overall goal must be that we maintain support for all Node versions we currently maintain until we drop support in the Next major (TBD). If it is possible to accomplish this with when updating to the v3 SDK we can do this. However, I can't guarantee any time frame, given that we're swamped with tasks and serverless is pretty low on our list of priorities. PRs are defiitely welcome here ;)

Furthermore, we'll need to look at

https://github.com/getsentry/sentry-javascript/blob/0b229bbe2d485fc5c45fb49418102866b5592bf3/packages/serverless/src/awsservices.ts#L43-L45

which patches a method of the AWS SDK. I don't know at the moment if this API changed in v3 and if/how we can patch both.

Thanks for the help!

[throrin19]

@Lms24 According to AWS documentation, AWS SDK v2 is available directly in lambda default layer in node version <= node14. For node >= 18 it's AWS SDK v3.

The problem is mainly in the optimization of the lambdas, and in particular the size of the lambdas.

Basically, the serverless framework won't integrate aws-sdk packages into the final build if they correspond to those provided by the default layer of the node version.

In our case, given that sentry-serverless uses sdk v2, if we decide to build on node <= 14, no worries, the sdk won't be included in the final code, we'll use the layer's sdk.

On the other hand, if you decide to build on node >= 18, you'll have to integrate the sdk v2 code, as it's not provided by default.

I don't know if I'm helping to clarify the problem.

[throrin19]

Any news about that ? Aws just launch node 20 compatibility with lambdas and, according to the documentation :

Up until Node.js 16, Lambda’s Node.js runtimes included the AWS SDK for JavaScript version 2. This has since been superseded by the AWS SDK for JavaScript version 3, which was released in December 2020. Starting with Node.js 18, and continuing with Node.js 20, the Lambda Node.js runtimes have upgraded the version of the AWS SDK for JavaScript included in the runtime from v2 to v3. Customers upgrading from Node.js 16 or earlier runtimes who are using the included AWS SDK for JavaScript v2 should upgrade their code to use the v3 SDK.

[lforst]

Hi, coming in with a few infos:

• ~The Sentry Serverless SDK doesn't require the aws-sdk at all. It's a dev-dependency. If it isn't installed the SDK should fail gracefully.~ Edit: This is wrong. The SDK uses aws-sdk but things should still not crash for v3.
• The Sentry Serverless SDK only interacts with the aws-sdk by monkeypatching the package's makeRequest function to instrument outgoing requests made with that function. Everything else should work as expected in the future.
• The Sentry SDK is compatible with Node 20.

[alfaproject]

@lforst not quite sure what your contribution to this thread is. AWS SDK v2 is deprecated, AWS is asking people to move away from it. It's not even bundled anymore in the latest runtimes. The node 16 layer is the last one where it is and they are deprecating that layer in a few months.

What's been asked here is for AWS SDK v3 support and not whether or not AWS SDK v2 support works. Of course if you don't use aws-sdk then everything works but that's what is deprecated.

[lforst]

@alfaproject Skimming this issue I didn't read it as asking for AWS SDK v3 support. I corrected my previous comment. The comment was intended to inform people coming to this thread with questions about compatibility because it seemed like @throrin19 was concerned about bundle size and compatibility in his comments.

As it stands aws sdk v3 support is currently not a big priority for us.

[alfaproject]

@lforst alright, fair enough, but do you or any other team member have any timeline on AWS SDK v3 support? We basically lost AWS support in Sentry for a few versions now because we don't use v2 anymore for the above reasons.

Of course, we can copy all the work you did and port it to SDK v3 but that sounds like a waste of resources for both of us if you are going to do it yourselves eventually. We can live a few more weeks/months without official support but it would be nice to have it.

[lforst]

@alfaproject We don't have a timeline and it doesn't seem to high priority considering the rather low amount of people asking for it. I don't see any work done as wasted. See yourself invited to contribute!

[alfaproject]

Sadly, we can't share any work done internally (hence 'waste of resources'), but if there's no timeline for official support then I guess we'll have to work on it then. Sad to hear but it is what it is.

[throrin19]

But the problem is that this choice makes using sentry with node 18 or 20 on AWS lambdas complex: larger build sizes and the use of a library deprecated by AWS.

What's more, support for node 16 lambdas is due to end in 2024, so not having it in the timeline is a real problem in terms of security and continuity with the new versions and respect for standards.

Wouldn't it be better to have a sedonc package in the sentry repo for node18+ lambdas using sdk v3?

Because of this I can't use sentry on my project even though I pay a subscription fee....

[lforst]

@throrin19 I don't understand. Sentry does not pull in the AWS SDK v2. It merely uses it if it is available. You can use the SDK even though we don't officially support the AWS SDK v3 yet. You can use whatever node version you like with the Sentry SDK (as long as it's Node version >= 8).

[throrin19]

@Ifost AWS SDK v2 is used by sentry and builded into lambda. After that we have the deprecation warning of AWS for this library. Addition size is not negligible : With lambda on node 18+ we can't ignore aws-sdk and import it directly in the code like we said in previous responses.

I don't understand why you don't follow the specs shared by AWS and use unmaintained/deprecated library for next node versions

[lforst]

@throrin19 Could you point me to the location in the SDK code where Sentry is pulling in the AWS SDK v2? I genuinely don't understand how this is a problem apart from not instrumenting the SDK v3.

[throrin19]

I think the problem load aws-sdk V2 in sentry is that line : https://github.com/getsentry/sentry-javascript/blob/93bbeb782ae588aee2d021a7eba3cd6d946e69be/packages/serverless/src/awsservices.ts#L44

[lforst]

@throrin19 but that line doesn't pull in the SDK, it merely requires it. It shouldn't have any effect on lambda size as you mentioned.

[throrin19]

If I remove sentry call, I don't have aws SDK v2 in my builded files

[lforst]

@throrin19 Ok, that means you have the AWS SDK v2 installed locally and your bundler is pulling it in. This is not something we can fix.

[throrin19]

you don't understand or don't want to understand: I don't have AWS SDK2 as a dependency in my project. the only time it gets added is when I use Sentry. so there is definitely a problem in your library which uses it in the background. This is what I have been pointing out for several months but you seem not to understand.....Le 11 déc. 2023 18:26, Luca Forstner @.***> a écrit : @throrin19 Ok, that means you have the AWS SDK v2 installed locally and your bundler is pulling it in. This is not something we can fix.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

[lforst]

@throrin19 Feel free to share a reproduction example where the AWS SDK v2 is pulled in and we will take a look.

[throrin19]

I created a project with the example here : https://github.com/Aquassay/test-sentry-serverless-aws-sdk

You can run analyze of a lambda with or without sentry to saw the difference. And the difference is like I said : with Sentry, we have an import of aws-sdk V2 by this code line : https://github.com/getsentry/sentry-javascript/blob/93bbeb782ae588aee2d021a7eba3cd6d946e69be/packages/serverless/src/awsservices.ts#L44

This is the analyze without Sentry on the lambda (In my example, in TS with middy) :

And the analyze with Sentry :

[lforst]

@throrin19 In your esbuild config you neither marked @sentry/serverless nor aws-sdk as external which is gonna pull everything into your bundle. This is not something we can prevent from within the SDK. This is addressed by configuring your application properly.

[throrin19]

I think you don't want understand....If you made it external, it's not included in the lambda.in node 18+, AWS SDK v2 is not included : we have error in usage if we called this.And sentry is never included with your response.

This occured with execution error.The solution is to push them inside a lambda layer but it's not a proper solution and it have a really big problem with the library for now

[AbhiPrasad]

Will be released with 8.0.0.