Lambda as ALB target in Cloudformation

[ssoulier]

Hi,

Could you please add the cloudformation and SAM the ability to specify an events for lambda which is an Application Load Balancer ?

Thanks.

[rsram312]

Went through the documentations. Not sure if it is updated either. Can you please add the SAM template ability to specify ALB feature for Lambda

[lorddelicious]

Yes please SAM ALB is needed. Discontinuing use of API Gateway due to high cost, transitioning to ALB.

[ssoulier]

Any update on this feature request ?

[kauphylover]

Another request to the AWS team to prioritize this!

[dbettin]

Another request. Thanks!

[brettstack]

Absolutely. At a minimum we could add a new Event for ALB which create the permission for you and you could specify your existing ALB. However, it's probably more likely that you want SAM to create the ALB target group for you, or maybe even the entire ALB (similar to how we create the API).

Would love to get some more feedback from everyone on this, including some SAM syntax.

[rsram312]

@brettstack Ideally, what I was hoping is if SAM can create the entire ALB and register Lambda as target. Probably following might be capabilities required:

1. Creating an ALB
2. Creating a target group with target type as Lambda with options to enable health check
3. Adding listeners to the ALB
4. Adding listener rules to the ALB
5. Adding permissions to the Lambda to access the ALB
6. Register the targets with the target group

I am not exactly sure which of the above features are already available in SAM. So, please correct me if anything above is redundant.

Currently, I was using a workaround by creating a Lambda function which does all these operations and was invoking that lambda wherever required. However, the feature being available in SAM would be really handy in directly leveraging it within the SAM template.

[brettstack]

@rsram312 that's very useful. Do you have an existing minimal template that includes all of those required resources?

[rsram312]

@brettstack The sample template I was using as workaround leverages boto3 to create those resources. If it might be of any use, I could probably send it over.

[deleugpn]

For me the preference is in specifying an existing ALB and getting the target group and permissions created by SAM.

[kauphylover]

I like what @deleugpn suggests - seems like the most natural way to go about it.

[brettstack]

We will try to accommodate all scenarios.

1. Create everything for you (ALB, Group, Permissions)
2. Create just Group and Permission
3. Create just Permission

[deleugpn]

I'm not sure if it would be possible, but I would guess that the following would be quite an amazing syntax:

      Events: 
        MyEventName:
          Type: ALB
          Properties:
            LoadBalancerArn: !ImportValue LoadBalancer
            ListenerArn: !ImportValue Listener
            CertificateArn: !ImportValue Certificate
            Condition: [...]

I don't know if I would need to specify anything else other than this.

The expected result would be something similar to this:

  HttpsListenerRule:
    Type: AWS::ElasticLoadBalancingV2::ListenerRule
    Properties:
      Actions:
      - Type: forward
        TargetGroupArn: !Ref TargetGroup
      Conditions: [Available on the Event Property]
      ListenerArn: [Available on the Event Property]
      Priority: [This is a tricky one. I think we have no other option than to delegate this to the user to define, unfortunately]

  TargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      TargetType: lambda

[luketn]

  Priority: [This is a tricky one. I think we have no other option than to delegate this to the user to define, unfortunately]

The valid range of priorities is from 1 - 50,000.

One approach we could take to this is to default the value to a random number 40,000 to 50,000 catch the exception of an already used priority and choose again.

As long as the path / domain is unique, the priority should not matter.

[deleugpn]

SAM would be limited to deploy up to 10,000 lambdas on a single Listener, I guess that's fine.

[ravibarkhani]

Hello, is SAM supports event from ALB now ? I tried to create a TargetGroup using cloudformation it fails in two way:

  ALBTargetGroup: 
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Name: !Ref LoadBalancerName
      VpcId: !ImportValue "VPCCreate-VpcId"
      Port : 80
      Protocol: HTTP
      TargetType: lambda

1. With below template it fails with error : "Port cannot be specified for target groups with target type 'lambda' (Service: AmazonElasticLoadBalancingV2; Status Code: 400; Error Code: ValidationError; Request ID: 3eaa3569-4b36-11e9-a442-1f94e3803149)"

2. When I do not provide port in above template it fails with message "Property Port cannot be empty."

[kylegordon]

This is still an issue. I get the same errors as @ravibarkhani

It's been 4 months since Lambda targets for ALBs was announced, and Cloudformation is lagging behind.

[keetonian]

You can use any cloudformation resource in your SAM templates; SAM does not alter these resources. You should be able to use any new feature from CFN in SAM without a SAM update unless an update is required to one of the AWS::Serverless::* resources.

If this doesn't work, it's an issue with either the configuration or CFN support, not SAM.

[s0enke]

CloudFormation got ALB->Lambda support in the last release (https://aws.amazon.com/about-aws/whats-new/2019/04/aws-cloudformation-coverage-updates-for-amazon-ec2--amazon-ecs-a/).

Here is a sample code how to use them: https://github.com/s0enke/cloudformation-templates/blob/master/templates/lambda-alb-sam-sample.yaml

[brettstack]

Thanks @deleugpn for the proposed syntax https://github.com/awslabs/serverless-application-model/issues/721#issuecomment-461555861 and @s0enke for the example. How might the input and output look for defining multiple paths/methods like we do for Api event? That is, we should allow defining the Rule for advanced configuration, but for common use-cases (e.g map this Path) we should provide simpler syntax which creates the necessary resources for you.

[petehanssens]

Thanks @s0enke for putting that together, I've gone one step further to add custom subdomain and auth0 authentication to the template, enjoy: https://www.peterhanssens.com.au/blog/2019-05/alb-to-lambda-with-auth0-authentication/

[metaskills]

@s0enke Thanks for that sample code. For Rails & Lambda using SAM, we were able to side step official support for this by just using CloudFormation. For the Lamby/Rails community, we had no need to for the sam local because Rails development is already easy and we use SAM as a means to define what we need in staging, production, etc.

https://github.com/customink/lamby/pull/37

So maybe this helps shape what is needed for SAM to support this? How much should SAM build? I feel these are two distinct questions below after going thru this exercise with Lamby.

• Syntactic sugar for CloudFormation?
• CLI sam local server mocking an Application Load Balancer?

[raaone7]

+1

[mneil]

I'm concerned that the example, along with any examples I've found, for how to use Lambda behind an ALB leaves the lambda open to execution from any loadbalancer on AWS. The examples in this rails app do not show how to use SourceAccount or SourceArn to restrict access. I have been unable to launch a lambda using either of those restrictions behind an ALB successfully. Could someone update the example, or provide an example, of how to use lambda behind an ALB that does not allow access to ~the world~ the entire account?

Edited: The lambda is access to the entire account, not the world

[raaone7]

Hey Michael,

you can use lambda behind lambda, using few methods

• alb subnets in private subnet
• restrict alb with WAF + ipwhitelist to your VPC and NAT cidr
• restrict alb with WAF + api gateway header check rule
• restrict the alb sg to your vpc cidr only

hope above helps.

regards, ram

On Sat, Aug 10, 2019 at 8:24 AM Michael Neil [email protected] wrote:

I'm concerned that the example, along with any examples I've found, for how to use Lambda behind an ALB leaves the lambda open to execution from any loadbalancer on AWS. The examples in this rails app do not show how to use SourceAccount or SourceArn to restrict access. I have been unable to launch a lambda using either of those restrictions behind an ALB successfully. Could someone update the example, or provide an example, of how to use lambda behind an ALB that does not allow access to the world?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/awslabs/serverless-application-model/issues/721?email_source=notifications&email_token=ACRZW65YFBNPKCFEG33RPELQDXVDPA5CNFSM4GKMLMN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD375PII#issuecomment-520083361, or mute the thread https://github.com/notifications/unsubscribe-auth/ACRZW6ZKNC574PM3GV7JFUDQDXVDPANCNFSM4GKMLMNQ .

[praneetap]

Thanks for the feedback @mneil! Could you send us some links to the examples you are referring to?

[luketn]

BTW due to the long timeframe on this I’m using Serverless for ALB/lambda projects: https://serverless.com/framework/docs/providers/aws/events/alb/

I’d suggest taking inspiration from their syntax - it’s pretty nice!

[deleugpn]

I also stopped recommending SAM as the go-to tool for serverless deployment. AWS is developing and marketing serverless A LOT but lagging behind on CloudFormation / SAM A LOT as well. 3rd party tools are doing a much better job at supporting AWS-provided features than AWS themselves.

[brysontyrrell]

Plus one from me on this.

I'm breaking up a stack currently and taking a set of my endpoints off an API Gateway and moving them behind an ALB. Based on another stack where I did an ALB->Lambda stack, I need to build the following resources into my template to make this work:

AWS::EC2::VPC (and all the stuff that goes with that like subnets, IGW, EIP, etc.)
AWS::EC2::SecurityGroup
AWS::Lambda::Permission
AWS::ElasticLoadBalancingV2::LoadBalancer
AWS::ElasticLoadBalancingV2::TargetGroup (per Lambda)
AWS::ElasticLoadBalancingV2::ListenerRule (per Lambda)

That's... a lot of stuff. Really, AWS needs to make ALBs a true serverless service and remove the requirements for an underlying VPC. This all becomes much simpler then. Until such time, SAM has an opportunity to abstract this boilerplate madness away in a simplified syntax.

[andrewdmay]

I'm concerned that the example, along with any examples I've found, for how to use Lambda behind an ALB leaves the lambda open to execution from any loadbalancer on AWS.

@mneil The only way I've found to restrict access to the Lambda to a specific target group is to name the target group and use a wildcard in the SourceArn on the Permission:

SourceArn: !Sub arn:aws:elasticloadbalancing:${AWS::Region}:${AWS::AccountId}:targetgroup/MyTargetGroupName/*

This allows the permission to be created before the Target Group (which otherwise fails because of the missing permission) and prevents a circular reference. Unfortunately it does mean naming the target group, which isn't idea, and you can't match on the full ARN.

You can modify the permission to add the SourceARN after the fact (and that could be done within a custom resource).

I took a look at what the Serverless Framework was doing for this because at first it appeared they had somehow worked around this problem - but actually what they are doing is first creating a permission without a SourceARN that the Target Group depends upon, and then they add a second permission that uses the SourceARN after the Target Group has been created. This doesn't increase security, but does make the ALB show up as the event source for the Lambda in the Console.

(which if you use the wildcard SourceARN, you get an error about it not matching a Target group, but it does appear to work, and I assume the restriction works, but I haven't verified that).