cli icon indicating copy to clipboard operation
cli copied to clipboard

Run prebuild pages in parallel

Open StarpTech opened this issue 6 years ago • 0 comments

Currently, I see no benefit to run the builds (pages) sequential. We can improve the performance by a factor of 5x if we run them in parallel. Or do I miss something?

Example

prebuild-worker.js

'use strict';

const { run } = require('@marko/prebuild');
const lassoConfig = require('./src/lasso-config');

async function prebuild(path) {
  await run({
    config: lassoConfig, // Either a lasso config object, or a path to one.
    flags: [], // Lasso flags to use when building the pages.
    pages: [path]
  });
}

prebuild(process.argv[2]).catch(err => {
  throw err;
});

prebuild.js

'use strict';

const globby = require('globby');
const execa = require('execa');

async function prebuild() {
  const ops = (await globby(['src/pages/*/*.marko'])).map(path =>
    execa('node', ['prebuild-worker.js', path])
  );
  await Promise.all(ops);
  // All templates have their prebuild.json files written to disk.
  console.log(`Prebuild completed!`);
}

prebuild().catch(err => {
  throw err;
});

Benchmark

CPU: AMD Ryzen 5 2600X (12) @ 3.600G SSD

12 pages, 30 components with sass, babel, minifying, bundling

~80sec
~17sec with the solution above

StarpTech avatar Apr 04 '19 20:04 StarpTech