Sam local Invoke lambda with local layer error

[Cecarlego]

Description:

When invoking a function locally, which relies on a layer built locally as well, the function cannot find the module which is part of the layer. When uploading that layer through the AWS console, and changing the Layer's reference in the template.yaml file, the import works. The layer is 17Mb zipped, and contains the mysql-connector-python module, as well as the requests module.

Steps to reproduce the issue:

1. Have a larger than 10MB layer (maybe more, just took the recommended max size from AWS console)
2. Import layer's package and run sam local invoke FunctionName
3. Deploy the layer manually in AWS console
4. Run sam local invoke FunctionName again after changing the lambda's layers property to the remote ARN in template.yaml.

Observed result:

Failure when calling Layer locally:

Expected result:

Calling layer locally or from remote have the same output.

[indyfin]

The layer ... contains ... the requests module.

@Cecarlego - why wouldn't you simply import from botocore.vendored import requests as provided by aws? I know this doesn't directly solve the problem but just wanted to make sure you knew about this.

[jfuss]

@indyfin DO NOT do that. The vendored requests library is old and not updated, see https://github.com/boto/botocore/issues/1745#issuecomment-497833200 and https://github.com/boto/botocore/issues/1608

[surbhi029]

I am also getting the same "unable to import package" error while invoking my lambda locally using 'sam local invoke FUNCTIONNAME'. Those packages are already present in layer(zipped folder). Is their any way to test these lamdas locally which are having python dependency zipped into layer.

Expectation is all the layer modules should get downloaded at some temp location while doing 'sam local invoke' so that it can take all the packages and run the lamda.

'apiconfig' module is already present in zipped layer.

[Cecarlego]

Hey @surbhi029, there is a way to test the lambdas locally when using layers, but it requires the layer to be deployed in your AWS account, and changing the reference in your template.yaml file to be the layer's arn rather than the local file path to it.

[jfuss]

@Cecarlego We support layers that are local as well, just not ones that are already zipped. The value of Content or ContentUri (depending on the resource you are using for Layers), should be the directory to where the code is and be unzipped. If that is true, then we will pick up that layer, combine it with any other layers for the function and make it available in the container.

[surbhi029]

Thanks a ton @Cecarlego @jfuss Both way it worked. I was stuck here from long back. Thanks again 👍

[XiamiYoung]

@Cecarlego We support layers that are local as well, just not ones that are already zipped. The value of Content or ContentUri (depending on the resource you are using for Layers), should be the directory to where the code is and be unzipped. If that is true, then we will pick up that layer, combine it with any other layers for the function and make it available in the container.

I have a dependencies folder that is unzipped, and be referred by ContentUri from template.yaml, however If install a custom module in dependencies by "file:my_modue_path", it's not working on Mac OS, but working on windows. Please check below: https://github.com/awslabs/aws-sam-cli/issues/1481

Thank you

[jfuss]

We don't follow paths in dependencies. Layer needs to be self contained, which is the same for the Lambda service.

[mikelax]

I just ran in to this issue. I was using a local zip file for the value of the ContentUri property within the LayerVersion. After switching this to a folder containing the contents of the zip file then a local invoke worked.

Is supporting zip files for a local invoke on the roadmap at all?

[awood45]

Tracking as a feature request to handle this type of layer source. For now, what local supports is a reference to the layer source.

[CJohnsonLehi]

I encountered this same problem. I spun up the docker image and connected via a bash terminal. What I found was that the lambda zip file was indeed copied to the /opt directory, but that it was NOT unzipped. Hence the failure.

[CJohnsonLehi]

I just ran in to this issue. I was using a local zip file for the value of the ContentUri property within the LayerVersion. After switching this to a folder containing the contents of the zip file then a local invoke worked.

The reason is as I mentioned above. The contents pointed to in ContentURI are simply copied over to the /opt directory, and not unzipped when using local invoke. I too can confirm that when I change the ContentUri value (In my template file): lambda-layers\python_files.zip to this: lambda-layers it indeed works.

[lambait01]

I do have the same issue, but layers dependencies are inside the requirements.txt. But not working, how can I do ?