Use Yarn instead of NPM with Typescript and Yarn workspaces

[gruckion]

Is your request related to a problem? Please describe.

We have a lot of lambda functions which are grouped into multiple nodejs project packages based on their business domain. One project might have 3 handlers and another may have 1. This is done due to some lambdas using identical dependencies based on business domain.

We use yarn workspace

Our nodejs project makes use of Yarn workspaces which helps with grouping common lambda functions into a single package. We also have a shared code package which the lambda packages reference (this would be a good use case for lambda layers). Workspaces also allow us to share a common directory and to run a script against all the lambda packages from the root of our project, e.g. running all tests, transpiling or deploying (via Serverless Framework). We're using webpack to transpile Typescript into Javascript for runtime

As such we can't run npm install and must use yarn install.

Describe the solution you'd like.

1. How can we use this package with yarn

2. How can we have multiple lambda handlers in a single npm project which gets packaged up as multiple zip files?

3. How can we run these lambda functions locally for testing and how can mocking of DynamoDB be handled in this context?

Describe alternatives you've considered.

We are considering npm_requirements = false, then providing commands = []to trigger the transpile step from typescript to javascript and have it produce multiplehandler.jsfiles and then loading each into thepath` variable.

There's an issue where than each usage of the module will need to produce the package zip for its respective handler. But each use will be running the transpile command to get the ALL individual handlers but only one will be zipped. This is a little wasteful, so looks like we will need ensure the transpile step can single out a specific handler of interest.

Additional context

I'm about to setup AWS StepFunctions to call lambda functions. The stepfunction module needs the lambda ARNs and our Lambda project needs a reference to the stepfunction ARN. So we have this chicken and egg problem. In order to deploy terraform we first need to create our lambdas (which is currently done using Serverless Framework). But the lambdas are reading from parameter store to get a reference to the step function ARN.

The only way we can do this at the moment is to temporarily remove the environment variables loaded from ssm parameter store, deploy the lambdas then apply Terraform infrastructure. Then deploy the serverless lambda functions a second time with the environment variables via ssm parameter store added back in. It's dirty.

Pros and cons

SFN lambdas in serverless

PRO:

1. We're already using Serverless

CON:

1. Can't provision the the state machine without a link to the lambdas
2. Can't provision the lambdas if we request the state machine arn from ssm which is in turn provided by terraform
3. Cloudformation isn't great slow and brittle
4. Forced to share variables between Terraform / Serverless using AWS Parameter store
5. We're not using Serverless for the rest of the infrastructure

SFN lambdas in Terraform

PRO:

1. Having all infrastructure exists in Terraform
2. No need to share variables outside of Terraform
3. Can directly share any Terraform variable as an environment variable with the lambda functions
4. Terraform is faster, can change infrastructure without having to publish an unchanged lambda.zip

CON:

1. Terraform module for packaging lambdas doesn't handle Typescript / Webpack so this will need to be a separate step
2. Terraform module for packaging lambdas doesn't use Yarn but does support executing any command