Metafile not working

[mrgrain]

Describe the bug The metafile option has no effect.

To Reproduce

new TypeScriptCode("src/index.ts", {
  buildOptions: {
    metafile: true
  }
});

Expected behavior I can access the metafile somehow.

Versions:

• 3.7.0

Additional context The JS API does return the file as a variable instead of writing a file. I'm not sure if what would be possible here, but ideally I could do something like this:

const code = new TypeScriptCode("src/index.ts", {
  buildOptions: {
    metafile: true
  }
});

console.log(code.metafile);

or at least have it write a metafile.json.

[nmussy]

Alright, so after spending some time trying to implement web cache busting, I've found a (convoluted) solution that does use the metafile option.

We can use a basic buildProvider to retrieve esbuild.buildSync()'s output on the fly:

import {
  BuildOptions,
  IBuildProvider,
  TypeScriptSource,
} from '@mrgrain/cdk-esbuild';
import {Metafile, buildSync} from 'esbuild';

let metafile: Metafile | undefined;
class MetafileEsbuild implements IBuildProvider {
  buildSync(options: BuildOptions): void {
    metafile = buildSync(options)?.metafile;
  }
}

new BucketDeployment(this, 'deployment', {
  sources: [
    new TypeScriptSource('../src/index.tsx', {
      buildProvider: new MetafileEsbuild(),
      buildOptions: {
        bundle: true,
        entryNames: '[name]-[hash]',
        metafile: true,
      },
    }),
  ],
  destinationBucket,
});

For my use-case, this then gets pretty janky. By chaining another BucketDeployment following this one, it is possible to read, modify and upload a templated index.html file. DeployTimeSubstitutedFile seems like it would be an ideal candidate, instead of a secondary BucketDeployment, but I don't think it's possible to give it an output key, it generates its own objectKey.

You would need to add a exclude: ['index.html'] property on the initial deployment, otherwise it would prune the file uploaded by the second

const getFilenamesFromMetafile = (metafile: Metafile) => {
  const entryPoint = Object.entries(metafile?.outputs ?? {}).find(
    ([, {entryPoint}]) => entryPoint?.endsWith('index.tsx'),
  );
  if (!entryPoint) throw new Error('Missing entryPoint');

  const [jsPath, {cssBundle: cssPath}] = entryPoint;
  if (!cssPath) throw new Error('Missing cssBundle');

  return {
    js: basename(jsPath),
    css: basename(cssPath),
  };
};

const fileNames = getFilenamesFromMetafile(metafile!);

new BucketDeployment(this, 'secondary-deployment', {
  sources: [
    Source.data(
      'index.html',
      readFileSync('../template/index.html', 'utf8')
        .replace('{{indexJsPath}}', fileNames.js)
        .replace('{{indexCssPath}}', fileNames.css),
    ),
  ],
  destinationBucket,
  prune: false,
});

AFAIK, since the TypeScriptSource is only built during the deployment phase, not the synth, there is no way to run the buildProvider before another source in the same deployment would be evaluated.

Another (somehow more reasonable solution) is to run esbuild once before the TypeScriptSource deployment, to obtain the metafile before feeding it into a singular BucketDeployment:

const buildOptions: BuildOptions = {
  bundle: true,
  metafile: true,
  entryNames: '[name]-[hash]',
};

const {metafile} = buildSync({
  ...buildOptions,
  entryPoints: ['../template/src/index.tsx'],
  absWorkingDir: resolve(__dirname, '..'),
  outdir: FileSystem.mkdtemp('esbuild'),
});
const fileNames = getFilenamesFromMetafile(metafile!);

new BucketDeployment(this, 'deployment', {
  sources: [
    new TypeScriptSource('../src/index.tsx', {buildOptions}),
    Source.data(
      'index.html',
      readFileSync('../template/index.html', 'utf8')
        .replace('{{indexJsPath}}', fileNames.js)
        .replace('{{indexCssPath}}', fileNames.css),
    ),
  ],
  destinationBucket,
});

Unfortunately, I have not been able to get the same hashes between the two back-to-back builds, even whilst running the same esbuild version, and giving it the same options as the internal ones (with the exception of the temporary outdir). I might be doing something wrong, given the claims of a stable metafile by evanw, but even if I got it working, maintaining hash parity between the pre-build and the internal one would become a concern during updates.

I'm very open to suggestions here, both solutions are clearly unsatisfactory. The best thing I can think of is adding a preBuild option to TypeScriptSource, that would generate an immediately usable metafile, and is then reused as a BucketDeployment source during deployment.

[mrgrain]

This is pretty cool, thanks for investigating!

Sound like the best option would be to have a new metafile getter, that will preemptively run the when requested and keep a cache so the the actual second build doesn't do additional work. But what I hear from you is that it will be a challenge to ensure the metafile build is the same as the synth one.

[nmussy]

Another possibility would be another implementation that runs asynchronously and returns both a "static" ISource to be fed to the BucketDeployment, and the esbuild BuildResult (which would include the metafile if the option is given). That might me the most versatile solution, and could satisfy #193

[mrgrain]

The issue with async is that we can't use await statements in a constructor, making it unusable for virtually all CDK app structures.