aws.lambda.CallbackFunction: Support lambdas greater than 69905067 bytes via intermediate S3 bucket

[joeduffy]

From @chrsmith on November 15, 2017 19:29

Running pulumi preview doesn't detect if a lambda to be deployed is too large for AWS, which leads to a catastrophic failure when you try to deploy your program. Perhaps we could issue a warning/error if we detect this during preview?

error PU2003: Plan apply failed: rpc error: code = Unknown desc = Error creating Lambda
function: RequestEntityTooLargeException: Request must be smaller than 69905067 bytes
for the CreateFunction operation

status code: 413, request id: 8ae171d3-ca34-11e7-b1aa-01928c26ca07

Copied from original issue: pulumi/pulumi#573

[joeduffy]

/cc @lukehoban I would assume we actually want to make this work somehow, perhaps by using S3 for the upload rather than streaming the zipfile directly. I've changed the title. If I'm mistaken, please let me know. I'm also moving to the AWS repo.

[lukehoban]

This has not come again in nearly 2 years - and users can of course move to using S3 directly if they want - closing for now.

[Taras-Tymchiy]

Hello. I'm currently getting similar error when trying to deploy my stack

error: 1 error occurred:
	* updating urn:pulumi:dev::aok-multi-pulumi::aws:lambda/function:Function::dev-v1-internal-reportProblem: 1 error occurred:
	* error modifying Lambda Function (dev-v1-internal-reportProblem-98cdf56) Code: RequestEntityTooLargeException: 
	status code: 413, request id: c5d75820-7a09-4859-a18a-991ac87c88b5

The weird thing about this error that it goes away if I run pulumi up again in few minutes. But then it would appear again when we try to deploy some other changes to that stack.

Lambda package size (after the latest deployment) is 33.0 MB.

Also, @lukehoban I have a question regarding

and users can of course move to using S3 directly if they want

Do you mean there is a setting in pulumi that enables uploading lambdas to s3? I that a documented feature?

Thanks in advance

[Taras-Tymchiy]

UPDATE:

The weird thing about this error that it goes away if I run pulumi up again in few minutes. But then it would appear again when we try to deploy some other changes to that stack.

Actually the lambda is not updated on the AWS, despite pulumi up not returning any errors.

[0xm1kr]

This has not come again in nearly 2 years - and users can of course move to using S3 directly if they want - closing for now.

@lukehoban I ran into this issue using APIGateway which uses the lambda function serialization aws.lambda.CallbackFunction can I use s3 in this instance?

[lukehoban]

I ran into this issue using APIGateway which uses the lambda function serialization aws.lambda.CallbackFunction can I use s3 in this instance?

Good point - it is not currently possible to use an S3 bucket with aws.lambda.CallbackFunction. I'll reopen this issue to track adding that support.

[SecurityRisk]

When trying to use an s3Bucket parameter with aws.lambda.CallbackFunction I'm met with this error

error: aws:lambda/function:Function resource 'callback-function-lambda' has a problem: Conflicting configuration arguments: "filename": conflicts with s3_bucket. Examine values at 'Function.Filename'.

[SecurityRisk]

If anyone runs across this issue and is trying to find a solution. A current workaround is to invoke an intermediate aws.lambda.Function (in this case a container Lambda function) inside your aws.lambda.CallbackFunction

const json = JSON.stringify({
    bucketId: bucket.get()
});
const lambdaResponse = await lambda.invoke({
    FunctionName: containerLambdaFunction.get(),
    Payload: json
}).promise();

I pass in ID's of resources that I want to use in the container so that I have access to them through the aws-sdk. You can create a container image the same way shown here. This way you can move all the heavy dependencies or large amounts of code to the container while still keeping the functionality of the aws.lambda.CallbackFunction.