Improve S3 Bucket Deployments for Web Apps

[amcdnl]

Description

Currently the CDKTF supports TerraformAsset which work nicely for resources like Lambda files, however, for static web application deploys to S3 its not ideal.

I've been attempting to do it something like this ( based on a few articles ):

    const files = glob.sync('../build/**/*', { absolute: false, nodir: true });
    for (const file of files) {
      new S3Object(this, `aws_s3_bucket_object_${path.basename(file)}`, {
        dependsOn: [bucket],
        key: file.replace(`../build/`, ''),
        bucket: bucket.bucket,
        source: path.resolve(file),
        etag: `${Date.now()}`,
        contentType: mime.contentType(path.extname(file)) || undefined
      });
    }

However, this approach is VERY slow since it has to catalog all the assets/etc and then manually upload one by one. When you have a fair amount of static assets like SVG icons or something, it can take 5+ minutes for a simple upload. I ended up moving this out of the process for the time being due to that but found the AWS CDK has a nice approach to handle this using a BucketDeployment construct:

        new deploy.BucketDeployment(this, "Deployment", {
            sources: [deploy.Source.asset("./build")],
            destinationBucket: siteBucket,
            distribution: siteDistribution,
            distributionPaths: ["/*"]
        });

Reference: https://dev.to/paulallies/deploy-your-static-react-app-to-aws-cloudfront-using-cdk-44hm

[ansgarm]

Hi @amcdnl 👋 That's a great idea. When building the serverless end to end example a while ago, I used the aws-cli directly to sync files to S3. But I agree, that we could support this via the CDKTF itself – maybe some kind of "deployment" construct might cover this in the future while allowing for different cloud provider specific implementations to work underneath.

@skorfmann did we have some kind of design issue for this broad topic already? I tried to find some, but didn't find anything besides the TerraformAsset feature, but that only copies / bundles files for use within the Terraform stage.

[amcdnl]

@ansgarm - Thanks for the quick response.

Thats literally one of the resources I found when trying to figure this out. I ended up using a Github Actions command to do this over a script:

    - uses: jakejarvis/s3-sync-action@master
      with:
        args: --acl public-read --follow-symlinks --delete

One other suggestion I wanted to make here is that TypeScript users are typically web devs and while I'm not new to this type of stuff by any means, it still took me several hours to get something going end-to-end. I feel like your really missing a huge community that would love this but it needs to be radically more simple to do simple things like deploy a create-react-app application and this could help. React even has a guide for different deployment models that I would encourage you to check out ( https://create-react-app.dev/docs/deployment ).

[skorfmann]

@skorfmann did we have some kind of design issue for this broad topic already? I tried to find some, but didn't find anything besides the TerraformAsset feature, but that only copies / bundles files for use within the Terraform stage.

Not that I'm aware of.

FYI: The mentioned BucketDeployment construct does some heavy lifting behind the scenes which would likely work quite differently on our end.

Would agree with @ansgarm here that some third party tooling is the best option for these kind of tasks at the moment.

[amcdnl]

@skorfmann -

Would agree with @ansgarm here that some third party tooling is the best option for these kind of tasks at the moment.

I found this article https://medium.com/scaffold/deploying-a-react-app-on-aws-s3-using-terraform-and-the-cdktf-eb7c91038879 which uses this product called Scaffold ( https://scaffold.sh/ ) but I don't really want 'yet another' thing in the mix to do a simple task. I was hoping CDKTF could help me eliminate these types of mundane scripting like this with abstractions for providers.

[ap0nia]

I did some poking around and I cobbled together something that seems to work with the current version of the CDK (TypeScript). The implementation is based on the official template-dir module.

import cdktf from 'cdktf'

// everything below inside of a stack

const bucket = new aws.s3Bucket.S3Bucket(this, 'bucket')

const fileTypes = new cdktf.TerraformLocal(this, "file-types", {
  ".css": "text/css; charset=utf-8",
  ".js": "application/javascript",
  ".json": "application/json",
  // ... and rest of types from the module, or any you can find
} )

const assets = new cdktf.TerraformAsset(this, 'assets', {
  path: 'path/to/static/assets',
  type: cdktf.AssetType.DIRECTORY,
})

const forEach = cdktf.TerraformIterator.fromList(cdktf.Fn.fileset(assets.path, '**'))

new aws.s3Object.S3Object(this, 'dynamic-assets-upload', {
    forEach,
    bucket: assets.bucket,
    forceDestroy: true,
    key: forEach.value,
    source: `${assets.path}/${forEach.value}`,
    etag: cdktf.Fn.filemd5(`${assets.path}/${forEach.value}`),

   // like the referenced module, every file is regex matched and the extension extracted;
   // the extension is used to lookup from the local variable, "fileTypes",
   // and defaults to "application/octet-stream" if not found
    contentType: cdktf.Fn.lookup(
      fileTypes.fqn,
      cdktf.Fn.element(cdktf.Fn.regexall("\.[^\.]+$", forEach.value), 0),
      "application/octet-stream"
    )
  }
)

It would be great to know if this was helpful in any way!