bref icon indicating copy to clipboard operation
bref copied to clipboard

Add first-party CDK construct

Open spaceemotion opened this issue 6 years ago • 8 comments

AWS' answer to infrastructure as code has been CloudFormation. Since that's kind of verbose to write they're working on CDK in which you write real code (js/ts, python ...) to define your resources (usually without having to define any roles or permissions). Under the hood this uses CloudFormation just as serverless does.

I've just converted my whole infrastructure to use CDK. Here's how I deploy a PHP lambda for laravel:

new lambda.Function(this, 'HTTP', {
  runtime: lambda.Runtime.PROVIDED,
  handler: 'public/index.php',
  code: lambda.Code.fromAsset('../backend'),
  timeout: cdk.Duration.seconds(28),
  memorySize: 1024,
  layers: [
    lambda.LayerVersion.fromLayerVersionArn(this, 'php-73-fpm', cdk.Arn.format({
      partition: 'aws',
      service: 'lambda',
      account: '209497400698', // the bref.sh account
      resource: 'layer',
      sep: ':',
      resourceName: 'php-73-fpm:11',
    }, this)),
  ],
  environment: {
    APP_STORAGE: '/tmp',
    LOG_CHANNEL: 'stderr',
    SESSION_DRIVER: 'array',
    VIEW_COMPILED_PATH: '/tmp/storage/framework/views',
  },
});

It would be great if bref could offer CDK constructs (which are their modular pieces, like plugins) that also reference the config file the serverless plugin uses.

spaceemotion avatar Sep 23 '19 07:09 spaceemotion

This is really interesting. It could totally replace serverless, isn't it? I thought myself multiple times that coding the infrastructure would be really nice, specially when building it using third party software.

Is that all the code necessary to deploy bref?

What does backend contain?

skyrpex avatar Sep 23 '19 14:09 skyrpex

@skyrpex Yes, I have recreated my full stack with CDK in a couple of hours (including waiting for about 40 mins for CloudFront to distribute haha).

My folder structure and set up looks as follows:

  • backend (laravel php backend)
  • frontend (vue cli SPA)
  • infrastructure (the CDK folder)

In about 200 lines of CDK I was able to get fully automated deployments with cache invalidation for the SPA running, the backend on an api.my.site subdomain including SSL certs for all of them (compare that to the over 1k of lines for the CloudFormation output).

The code pasted above is all the code you need to deploy bref as a lambda, albeit just the fpm version. I haven't added the console variant yet.

spaceemotion avatar Sep 23 '19 14:09 spaceemotion

It would be great if bref could offer CDK constructs (which are their modular pieces, like plugins) that also reference the config file the serverless plugin uses.

Hi, Bref layers are listed in this file: https://github.com/brefphp/bref/blob/master/layers.json

What about including that file in your JS CDK file to get the latest layer versions? Or would you need something else?

mnapoli avatar Sep 26 '19 08:09 mnapoli

That file is indeed necessary to offer constructs, but I think we should discuss what level of modularity we should offer with this CDK plugin, and also how to retrieve that layers.json file.

Level of Modularity

The Bref package for CDK could offer different methods. For example, a method to retrieve layers would be very simple:

// Version could be optional, so omitting it would require to download the layers.json file and use the stack.region.
const getBrefLayer = (stack: cdk.Stack, id: string, version: string) => lambda.LayerVersion.fromLayerVersionArn(stack, id, cdk.Arn.format({
  service: 'lambda',
  account: '209497400698', // the bref.sh account
  resource: 'layer',
  sep: ':',
  resourceName: `${id}:${version}`,
}, stack));

In order to use that layer, the user would need to write the following code:

const apiLambda = new lambda.Function(this, 'HTTP', {
      runtime: lambda.Runtime.PROVIDED,
      handler: 'public/index.php',
      code: lambda.Code.fromAsset(path.resolve(__dirname, '../api')),
      timeout: cdk.Duration.seconds(28),
      memorySize: 1024,
      layers: [
        getBrefLayer(this, 'php-73-fpm', '11'),
      ],
      environment: {
        APP_STORAGE: '/tmp',
        LOG_CHANNEL: 'stderr',
        SESSION_DRIVER: 'array',
        VIEW_COMPILED_PATH: '/tmp/storage/framework/views',
      },
    });

    const api = new apigateway.LambdaRestApi(this, 'myapi', {
      handler: apiLambda,
    });

If we were to provide higher level methods, such as a method to create a preconfigured new lambda.Function, we would need to decide what parts should be customizable or not. For example, this parameter should be definitely customizable: code: lambda.Code.fromAsset(path.resolve(__dirname, '../api')),

How to retrieve the layers.json file

The least intrusive option would be to dynamically download the layers.json file when the package constructs the layers/functions, but would send requests often.

Another way is to bundle the layers.json file in the npm package itself, but that would require some sort of automation to maintain the Bref PHP package and the Bref CDK package.

skyrpex avatar Sep 27 '19 10:09 skyrpex

@skyrpex that's the kind of things I was thinking about as well. Even better would be constructs specifically built for Laravel, Symfony, Zend... so people can get started quite easily.

spaceemotion avatar Sep 27 '19 11:09 spaceemotion

Alright, so since it's Hacktober, I thought about working on the following things for bref regarding CDK integration:

  • Provide an official construct library (pretty sure that this will have to be in its own repository I am afraid, they need compilers to transpile the code into c#, java and such).
  • Constructs offered initially are:
    • Basic PHP Lambda
    • Lambda for Laravel (just setting things like php path and basic env settings)

Not sure if the "config.json" can (or should) be extracted to its own package so the constructs don't have to be updated when layer versions change.

spaceemotion avatar Oct 06 '19 18:10 spaceemotion

If anyone wants to take ownership on this project let me know.

mnapoli avatar Nov 21 '20 17:11 mnapoli

Sorry for not taking this up on the last hacktoberfests. Just didnt have the time to tackle this in my free time 🙈

I can say though that AWS CDK is no longer the only option. There is also a Terraform Version that implements the same interfaces (curiously enough). Not sure if both should be supported then.

spaceemotion avatar Nov 21 '20 18:11 spaceemotion

@mnapoli

I am interested in creating this CDK Construct. Since we now have a layers.js package, this workflow is much easier and complete.

We can provide constructs that allow something like following. Adapted from above message.

const apiLambda = new bref.PHP74Function(this, 'HTTP', {
      handler: 'public/index.php',
      code: lambda.Code.fromAsset(path.resolve(__dirname, '../api')),
      timeout: cdk.Duration.seconds(28),
      memorySize: 1024,
      environment: {
        APP_STORAGE: '/tmp',
        LOG_CHANNEL: 'stderr',
        SESSION_DRIVER: 'array',
        VIEW_COMPILED_PATH: '/tmp/storage/framework/views',
      },
    });

    const api = new apigateway.LambdaRestApi(this, 'myapi', {
      handler: apiLambda,
    });```

gnanakeethan avatar Nov 08 '22 23:11 gnanakeethan

@gnanakeethan FYI I have started working on constructs, but I'd love to get contributors in the new project! I'll try to publish that hopefully next week, would you be interested to review and contribute?

mnapoli avatar Nov 09 '22 10:11 mnapoli

Here is the repository for CDK constructs: https://github.com/brefphp/constructs

mnapoli avatar Dec 16 '22 14:12 mnapoli

I have added some documentation to the repository, feedback is welcome: https://github.com/brefphp/constructs

I will consider this issue "done", even though I plan to add more constructs in the near future.

mnapoli avatar Feb 16 '23 16:02 mnapoli