Add AWS Lambda Support

[djonser]

Enables describing AWS Lambda Functions so that other tools can build upon this for things such as local orchestration, bundling, deployment and more.

This draft PR is about the general concept and the implementation goes far enough to make it work within the playground, but it's not complete or necessarily aligned. I hope it's enough for a discussion to find out what would be required to accomplish this. I've no affiliation with AWS.

Executable Project Functions

A regular project described as a Lambda Function.

var builder = DistributedApplication.CreateBuilder(args);

var project = builder.AddProject<Projects.LambdaRestApi>("api");
project.AsLambdaFunction("apiFunction");

{
  "resources": {
    "api": {
      "type": "project.v0"
    },
    "apiFunction": {
      "type": "aws.lambda.function.v0",
      "path": "../LambdaRestApi/LambdaRestApi.csproj",
      "handler": "LambdaRestApi",
      "runtime": "dotnet8",
      "project": "api"
    }
  }
}

Class Library Handlers

namespace MyProject;

public class OrderFunction
{
    public async Task HandleAsync(SQSEvent input)
    {
        // ...
    }
}

var builder = DistributedApplication.CreateBuilder(args);

var awsResources = builder.AddAWSCloudFormationTemplate("AspireSampleDevResources", "app-resources.template")

builder.AddLambdaFunction<LambdaFunctions.MyProject_OrderFunction_HandleAsync>("orderFunction");

{
  "resources": {
    "orderFunction": {
      "type": "aws.lambda.function.v0",
      "path": "../MyProject/MyProject.csproj",
      "handler": "MyProject::MyProject.OrderFunction::HandleAsync",
      "runtime": "dotnet8"
    }
  }
}

Code Generation

AWS .NET Class Library Handlers requires knowing the name of ASSEMBLY, TYPE and METHOD which is then composes the handler as seen above. By generating metadata similar to Projects there's type-safety since the AppHost will fail to build when non-existing/invalid Lambda functions are referenced.

namespace LambdaFunctions;

public class MyProject_OrderFunction_HandleAsync : global::Aspire.Hosting.AWS.Lambda.ILambdaFunctionMetadata
{
    public string ProjectPath => """/path-to/src/MyProject/MyProject.csproj""";
    public string Handler => "MyProject::MyProject.OrderFunction::HandleAsync";
    public string AWSEventBaseType => "Amazon.Lambda.SQSEvents.SQSEvent";
}

Implementation

When an AppHost ProjectReference has the property <AWSProjectType>Lambda</AWSProjectType> (docs) its forwarded to AWSLambdaMetadataGenerator that has a look inside the assembly for any methods that matches the criteria of a Lambda handler and then generates relevant metadata.

If the project is an executable, metadata that implements IProjectMetadata is also generated so that the handler will reflect the AssemblyName.

Summary

While the type-safety derived from describing Lambda Functions via generated metadata can be utilized in any tool that parses the manifest, my initial motivation was for a "shared/integrated experience" with AWS CDK (example below) where the Aspire AppHost and CDK Application coexist with convenience extensions available.

builder.AddLambdaFunction<LambdaFunctions.MyProject_Publisher_HandleAsync>("publisher")
    .WithEnvironment("QUEUE_URL", queue.QueueUrl)
    .CreateCDKFunction(scope, "PublisherFunction", function => { queue.GrantSendMessages(function); });

builder.AddLambdaFunction<LambdaFunctions.MyProject_Consumer_HandleAsync>("consumer")
    .WithEnvironment("BUCKET_NAME", bucket.BucketName)
    .CreateCDKFunction((handler, code, runtime) =>
    {
        var function = new Function(scope, "ConsumerFunction", new FunctionProps
        {
            Handler = handler,
            Code = code,
            Events = [new SqsEventSource(queue)],
            Runtime = runtime,
            MemorySize = 1024
        });
        
        bucket.GrantReadWrite(function);
        
        return function;
    });

Microsoft Reviewers: Open in CodeFlow

[normj]

I'm a little confused on the end goal. Is it to get the deployment information for the Lambda function is in the manifest and not supporting local development? We would then need some tooling update somewhere, possibly Amazon.Lambda.Tools to read the manifest and drive deployment.

When I have thought about supporting Lambda with Aspire my mind has been how could we get Amazon.Lambda.RuntimeSupport and some of the Lambda runtime API for the .NET Lambda Mock Test Tool to work together with the F5 debugging experience.

[djonser]

@normj

The end goal is both local development and deployment information.

This PR only focuses on providing a way to express our Lambda Functions in an "Aspirey-way" so that any tool can build upon this for any purpose. I believe this to be the first step.

For all examples builder is DistributedApplication and not an IResourceBuilder<IResource>

Deployment Tools

Now that the minimum required configuration of Lambda functions are known (source location, handler, runtime), tools can integrate with this information both within Aspire and externally (reading the manifest).

Amazon.Lambda.Tools could be updated to read the manifest or additional extensions in Aspire.Hosting.AWS for example builder.ForAWSLambdaTools(), could add a lifecycle that create/update aws-lambda-tools-defaults.json for expressed Lambda Projects on publish operations.

Maybe an extension such as builder.ForAWSSAM("../../template.yaml") could match on resource names and keep Handler, Runtime and CodeUri in sync.

Whether files other than the manifest should be created/updated on a manifest publish operation is a good idea or not I don't know.

Local Development

Local development for Lambda functions in Aspire would be a big win. An extension looking something like builder.WithAWSLambdaRuntime(config => {}) to enable runtime support for expressed Lambda Functions in the way AWS recommends and leaving the door open for builder.WithCustomThirdPartyAWSLambdaRuntime().

Add to this a sensible way to configure runtime inclusion/exclusion, for example builder.AddLambdaFunction().ExcludeFromLocalRuntime() and developer-overrides via EnvVars/appsettings.local.json.

Summary

I can see the confusion. End result of my implementation is information that just sits there in the manifest. I did not want to attempt too much at once and I believe that moving past the developer having to manually specify handlers such as Company.ShoppingFunctions::Company.ShoppingFunctions.OrderFunction::HandleAsync is a good foundation.

Also the discussion. Do you think something like this could be useful?

[martincostello]

When I have thought about supporting Lambda with Aspire my mind has been how could we get Amazon.Lambda.RuntimeSupport and some of the Lambda runtime API for the .NET Lambda Mock Test Tool to work together with the F5 debugging experience.

This would definitely be a welcome addition 👍

[normj]

@djonser I'm traveling till middle of next. When I'm back I can take a deeper look. I like the source generation of the function handlers in the AppHost making their values be checked by the compiler. The event source mapping has me worried in that it might imply some local dev support that is not going to be possible. For example if I create an AppHost which creates a queue and Lambda function users could expect when they F5 debug and then add a message to the new queue the function will be called within the local dev environment. I don't see that technically happening and that could really confuse users.

Let me get back to you next week when I'm back in the office.

[djonser]

@normj

I agree about event sources and have removed it. If a custom local development-runtime can support event sources then that package can introduce extensions for it instead.

I added a basic builder.AddAWSLambdaToolsSupport() so that a feature like this could be tried out. If the goal was to ensure dependent manifests are kept in sync when publishing the aspire manifest, how would this best be integrated?

Have a good time traveling.

[djonser]

API Update and Local Development

var builder = DistributedApplication.CreateBuilder(args);

// Executable Projects
builder.AddLambdaFunction<LambdaFunctions.LambdaRestApi>("api");
builder.AddLambdaFunction<LambdaFunctions.ExecutableLambdaHttpApi>("executableApi");

// Class Library
builder.AddLambdaFunction<LambdaFunctions.ClassLibraryFunctions_OrderFunction_HandleAsync>("orderFunction");

// Non .NET Runtime or .NET not part of solution
builder.AddLambdaFunction("nodeFunction", LambdaRuntime.Custom("nodejs20.x"), "index.handler", 
    "../../packages/node-app");

builder.Build().Run();

Metadata

There's now a single metadata type. It now contains OutputPath and Traits to aid in local development.

namespace LambdaFunctions;

public class LambdaRestApi : global::Aspire.Hosting.AWS.Lambda.ILambdaFunctionMetadata
{
    public string ProjectPath => """/path-to/playground/AWSLambda/LambdaRestApi/LambdaRestApi.csproj""";
    public string Handler => "MyRestApi";
    public string OutputPath => """/path-to/artifacts/bin/LambdaRestApi/Debug/net8.0/""";
    public string[] Traits => ["IsExecutableProject", "Amazon.Lambda.AspNetCoreServer.Hosting"];
}

Local Development

Starting the AppHost will now launch one dotnet-lambda-test-tool-x.x for each project that has expressed at least one Lambda Function. It's possible to interact with all functions but only executable projects are debuggable. I've not tested this in all combinations of OS and IDEs.

If the IDE attaches to the dotnet-lambda-test-tool process corresponding to a class library debugging becomes possible. If the IDE/IDE-Plugin started the test-tool as specified in documentation, debugging also becomes possible. Any guidance for how to achieve what would work the best cross-platform and IDEs would be appreciated.

Misc

• When executable project has a reference to package Amazon.Lambda.AspNetCoreServer.Hosting, no Mock Tool is started for the project.
• When using Mock Tool combined with central artifacts (as in this solution), aws-lambda-tools-x.json and serverless.template has to be copied to OutputPath.
• Disabling the Mock Tool can be done globally or on project-by-project basis.
• Any ProjectResource created on behalf of a LambdaFunction is currently hidden from the user and manifest. Projects only facilitates running the executable Lambda Function.
• TODO: A lifecycle hook will hide LambdaFunctions in the dashboard or set them as running with an endpoint pointing in the right direction.
• TODO: Executable Assembly Projects should wait for Mock Tool to start.

[djonser]

@normj , have you had any time to look at this?

To summarize:

• Generate metadata from referenced Lambda Projects for Lambda Function annotation in the AppHost and enable handler type-safety.
• Use same metadata and Aspire to launch Lambda Mock Tool for the annotated Lambda Functions.