Adding the ability to prefix assets with a CDN url

[MoaathAlattas]

Summary

Adding the ability to prefix assets urls with a string for static files (./static) and internal files (./_frsh/js/). This adds the ability to set a CDN url and load those files outside of fresh server.

Changes

• [x] Add a new config key cdnUrl
• [x] Add ASSET_PATH_PREFIX in a similar fashion to BUILD_ID
• [x] Add a runtime utility to determine static file path staticFile
• [x] Skip building static and islands js routes if cdnUrl is defined.
• [x] Add tests

Why

I am not using Deno deploy as I am hosting my app on one region closer to the database. However, I would like to host all static files on a CDN closer to the end user. There might be other benefits but this was it for me.

How It works

1. Set the cdn url as part of the config passed to the start function (./main.ts)

await start(manifest,{
  ...config,
  cdnUrl: "https://my-cdn-url.com/my-bucket-name"
})

2. Build your islands Ahead-of-time Builds
3. Create a script to upload ./fresh and ./static folders to your CDN, see an example below.
4. Add the script as a pre-deploy step, similar to the one here: Deploying an optimized Fresh project
5. Your app will prefix static and island js files with a cdnUrl/[build_id] value:

/_frsh/js/d763ab9889e57eba8c364daf5de8367e4746e61d/main.js
=>
https://my-cdn-url.com/my-bucket-name/d763ab9889e57eba8c364daf5de8367e4746e61d/_frsh/js/main.js

/favicon.ico =>
https://my-cdn-url.com/my-bucket-name/d763ab9889e57eba8c364daf5de8367e4746e61d/favicon.ico

Upload to CDN script (example)

This is an example to upload ./fresh and ./static folders to AWS S3 (or any compatible service such as cloudflare R2)

// deno run -A ./upload-to-cdn.ts
import { INTERNAL_PREFIX } from "$fresh/runtime.ts";
import { JS_PREFIX } from "$fresh/src/server/constants.ts";
import {
  PutObjectCommand,
  S3Client,
} from "https://esm.sh/@aws-sdk/[email protected]";
import { walk } from "https://deno.land/[email protected]/fs/walk.ts";
import { contentType } from "https://deno.land/[email protected]/media_types/content_type.ts";

const appAbsolutePath = Deno.cwd()
const cdnFolderPath = "/" + await getBuildID();
const BUCKET_NAME = "simple-pages";

const client = new S3Client({
    region: "auto",
    endpoint: '',
    credentials: {
      accessKeyId: '',
      secretAccessKey: '',
    },
  });

await uploadFolder(
  appAbsolutePath + "/_fresh",
  cdnFolderPath + INTERNAL_PREFIX + JS_PREFIX,
);
await uploadFolder(appAbsolutePath + "/static", cdnFolderPath);

// functions
async function getBuildID() {
  try{
    const { default: json } = await import(
      `${appAbsolutePath}/_fresh/snapshot.json`,
      { with: { type: "json" }}
    );
    return json.build_id;
  } catch (error) {
    console.log("did you build your app islands? ")
    console.log(error)
    Deno.exit(1)
  }

}

async function uploadFile(filePath: string, objectKey: string) {
  const fileContent = await Deno.readFile(filePath);
  const fileContentLength = fileContent.byteLength;
  const fileContentType = contentType(filePath.split(".").pop()!)?.split(
    ";",
  )[0];

  const command = new PutObjectCommand({
    Bucket: BUCKET_NAME,
    Key: objectKey,
    Body: fileContent.byteLength === 0 ? new Blob([""]) : fileContent,
    ContentType: fileContentType,
    ContentLength: fileContent.byteLength === 0
      ? new Blob([""]).size
      : fileContentLength,
    ACL: "public-read",
  });

  try {
    await client.send(command);
  } catch (error) {
    console.error(`Failed to upload: ${objectKey}`);
    console.error(error);
    Deno.exit(1);
  }
}

async function uploadFolder(fromPath: string, toPath: string) {
  console.log(`Uploading folder ${fromPath} to the cdn (${toPath}/)...`);
  for await (const file of walk(fromPath)) {
    if (file.isFile) {
      await uploadFile(file.path, toPath + "/" + file.name);
    }
  }
  console.log(`"Done uploading folder ${fromPath} to the cdn (${toPath}/)`);
}

[MoaathAlattas]

@marvinhagemeister added tests and left comments inline for new updates since last time. Let me know if there is anything else needed.

[MoaathAlattas]

One thing I was wondering while going through the changes if there is a way to combine staticFile with asset. It seems like the intention of the CDN url prefix is to be sort of a global flag for assets and static files alike. That way you could easily switch between using a CDN or not, via a config flag without having to go through the actual code. What do you think?

That's a good point, separating asset and staticFile doesn't feel right and could be confusing. I did combine them initially then I found out asset is applied automatically on src and srcset attributes in <img> and <source> tags only which doesn't cover other static files. In case of using a cdn url you have to use asset explicitly for other static files to prefix it with the cdn url.

Also, I found a case where some applications, instead of uploading files manually, they just use a proxy on another url. Then use the proxied url to serve static files (files will be cached on the first hit). This wouldn't work with the current changes because we changed assets paths and disabled static and internal routes.

I will think about both for a bit and see what would be the best way without changing the current behavior.