AWS lambda cross-platform issue

[tappmarket]

I am working on AWS lambda but my dev machine is Windows. I zip the node_modules and upload the package into Lambda layer. The problem is that I cannot use better-sqlite3 due to error

/opt/nodejs/node_modules/better-sqlite3/build/Release/better_sqlite3.node: invalid ELF header at Object.Module._extensions..node

I think it is because AWS Lambda is using Linux environment but the uploaded better-sqlite3 is Win version. Any suggestion to solve that? Please do not consider changing my dev machine to Linux

[JoshuaWise]

better-sqlite3 is a native add-on for Node.js, written in C++. You are compiling a native binary on one platform, and trying to run it on another platform. My recommendation would be to compile (npm install) on the same platform that will be running your program.

[JoshuaWise]

Other people seem to have run into the same issue with AWS Lambda:

• https://docs.aws.amazon.com/lambda/latest/dg/nodejs-package.html

  • If your deployment package contains native libraries, you can build the deployment package with AWS Serverless Application Model (AWS SAM). You can use the AWS SAM CLI sam build command with the --use-container to create your deployment package. This option builds a deployment package inside a Docker image that is compatible with the Lambda execution environment.

• https://serverlessguru.medium.com/how-to-run-native-addon-modules-for-node-js-in-aws-lambda-aa74e1a9010

[Prinzhorn]

There are also lambda layers, first search result https://github.com/seanfisher/better-sqlite3-lambda-layer (obviously outdated)

[JoshuaWise]

Closing due to inactivity

[fuadop]

Hi there @tappmarket, try out this my latest side project https://nodelayer.xyz/ to build a lambda layer to attach to your lambda function. It'll most likely solve compatibility issues between your machine and the lambda runtime.

[mnai01]

For those using SAM CLI I've tried a lot of different methods but what worked for me was simply just npm i better-sqlite3 in the most parent folder. If you pulled the AWS Serverless API template from sam cli init then you should also have that package.json there with aws-sdk added. I assume this just adds better-sqlite3 to every lambda as a layer. I have tried other methods with adding an AWS::Serverless::LayerVersion resource to your cloudformation template and installing node_modules in a separate folder then pushing that through GitHub actions but I'd get the same invalid ELF header error so this simple fix works for now

[pjvds]

Hi there @tappmarket, try out this my latest side project https://nodelayer.xyz/ to build a lambda layer to attach to your lambda function. It'll most likely solve compatibility issues between your machine and the lambda runtime.

FYI: when I use nodelayer.xyz for better-sqlite3, all I get is an empty layer.

[fuadop]

Hi there @tappmarket, try out this my latest side project https://nodelayer.xyz/ to build a lambda layer to attach to your lambda function. It'll most likely solve compatibility issues between your machine and the lambda runtime.

FYI: when I use nodelayer.xyz for better-sqlite3, all I get is an empty layer.

I generated the attached layer using this settings and it worked.

layer.zip

[pjvds]

@fuadop that seems to work indeed. It only gives me an empty layer if I picket only the v18 target.

What is interesting is that the layer doesn't build from source if I'm correct, since using the layer has a mismatch in libs, like glibc 2.29 instead of the one that the lambda amazin linux is running, glibc 2.34. Is that correct?