Allow packaging of static files alongside binary

[dancoates]

💡 Feature description

It would be nice if it was possible to use the include and exclude package configuration in serverless.yml to choose other files to bundle with the bootstrap binary. For example to include a .env file that is then read by the dotenv crate at runtime.

I am sure that there are other use cases for including files other than the binary but that is the one that sprang to mind.

[softprops]

That's a good point. I've deferred in adding features until they are needed or asked for. I'll look into this

[nosideeffects]

I was under the impression that this was already supported. How would I add this capability myself for my current serverless project?

[nickspell]

An alternative solution to the static file packaging problem is to use AWS Lambda Layers. Just put your static files into some directory at the root of your project and create a layer from that.

Example:

myservice
├── static
│   ├── foo.json
├── src
├── serverless.yml

In your serverless.yml insert:

layers:
  mydata:
    path: static
    description: My static json data

In your function you can put the dependency on your layer as a title cased reference:

functions:
  myfunction:
    handler: #...
    layers: 
       - {Ref: MydataLambdaLayer}
     # .....

You can now access your files as if they are located in the "/opt" folder of your lambda, e.g.:

let file = File::open("/opt/foo.json")?;

This solution is quite useful especially when you want to share your static data between different lambdas.

[cecton]

I think we could use the "include" attribute of a function and "add" the files to the zip after the build in the container succeeded.

[vim-zz]

Adding another use case, which requires having a certificate file to communicate with a certain server.

[sachaarbonel]

I just wanted to point out that the workaround of @nickspell doesnt work for my use case which is opening a sqlite db. For some reason I'm getting a SqliteError "attempt to write a readonly database". I guess the /opt folder is read only. EDIT : nevermind I copied the file to the /tmp folder (which is the only writable folder in aws lambda) and it did the trick.

[softprops]

I think the most appropriate way to approach this is the suggestion original mentioned. Use the built in include / exclude packaging options.

[kahlil29]

@softprops Could you tell me if it would be possible to use a rust program that has an OpenCV dependency? I'm currently using this crate Any thoughts about this? I'm guessing I need to introduce the opencv files (statically linked?) into the Lambda somehow?

[softprops]

This looks a bit involved but possibly doable. A lambda layer might be a useful approach here. Heres an example from the python world of setting up a layer https://www.bigendiandata.com/2019-04-15-OpenCV_AWS_Lambda/

I'm not familiar with the opencv project enough myself to know why there might not be an off the shelf layer available https://github.com/mthenw/awesome-layers

[kahlil29]

Thanks for the informative and helpful reply @softprops I had tried earlier (not using Layers) but could not get it working. I guess it's time to give Lambda Layers a go. So in your experience, it's common/acceptable to use Lambda Layers to package in dependencies for Lambdas? It's known to work without (many) issues?

[softprops]

Without more expertise in opencv I cant gaurantee this will work but to answer the question about using layers to package host dependencies, that's almost the sole use of layers https://aws.amazon.com/blogs/aws/new-for-aws-lambda-use-any-programming-language-and-share-common-components/

That said this is likely an easier case for interpreted languages im not sure if you can package and dynamically link via env var with rust. I glanced through the opencv docs and there was some suggestions that you could

[kahlil29]

Thanks a lot @softprops Appreciate the answer and info 😄