chatGPTBox icon indicating copy to clipboard operation
chatGPTBox copied to clipboard

Refactor build pipeline for performance and dev output reliability

Open PeterDaveHello opened this issue 2 months ago • 47 comments

User description

Replace JS/CSS minifiers with esbuild to reduce build times while preserving outputs.

Enable thread-loader by default in dev and production. Choose workers dynamically (based on CPU cores) and allow overrides via environment variables. Keep filesystem cache enabled and make cache compression configurable, defaulting to uncompressed for faster warm builds on CPU-bound machines. Add BUILD_PARALLEL toggle (default on) to switch between parallel and sequential production variant builds. Ensure watch-once dev builds exit cleanly.

Adopt sass-embedded for SASS processing. In development, use style-loader to speed up CSS/SCSS compilation while keeping production outputs unchanged. Maintain CSP-safe source maps for extension contexts and suppress CSS 404 noise in development outputs.

Additionally, dependency caching has been added to the GitHub Actions workflow to accelerate CI/CD runs.

Results on this DO VPS (2 cores, ~4 GiB RAM):

  • Production first run (cold): ~44s (baseline ~105s)
  • Production second run (warm): ~19s (baseline ~39s)
  • Development first run: ~31s; second run: ~29s

Times vary by environment; numbers above are for this machine.

Pull request summary from GitHub Copilot:

This pull request introduces several significant improvements to the build process, focusing on performance, configurability, and developer experience. The main changes include switching to faster build tools, adding advanced caching and parallelization options, updating build scripts, and improving CI build efficiency. These updates should result in faster builds, better resource utilization, and more flexibility for both local development and CI environments.

Build Performance and Tooling Improvements

  • Switched JS minification from Terser to esbuild (via esbuild-loader) and enabled esbuild-based CSS minification for much faster production builds. (build.mjs, package.json) [1] [2]
  • Added support for parallel builds of production variants, controlled via the BUILD_PARALLEL environment variable, and enabled thread-loader for Babel by default (configurable via BUILD_THREAD and BUILD_THREAD_WORKERS). (build.mjs, .github/copilot-instructions.md) [1] [2] [3]
  • Introduced advanced Webpack cache options, allowing filesystem cache compression to be toggled with BUILD_CACHE_COMPRESSION for improved warm build performance. (build.mjs, .github/copilot-instructions.md) [1] [2]

Build Script and Output Enhancements

  • Updated the build script to support both parallel and sequential builds, improved output directory management, and ensured correct copying of build artifacts for both production and development. (build.mjs) [1] [2]
  • In development mode, the script now creates placeholder CSS files to prevent 404 errors in the browser. (build.mjs) [1] [2]

Continuous Integration (CI) Improvements

  • Added npm and Webpack cache steps to the pre-release GitHub Actions workflow to speed up CI builds. (.github/workflows/pre-release-build.yml)

Dependency Updates

  • Added new dependencies: esbuild, esbuild-loader, sass-embedded, style-loader, and thread-loader; updated sass and sass-loader to latest versions; removed terser-webpack-plugin as it's no longer used. (package.json) [1] [2]

Documentation

  • Expanded the build instructions to document all new performance-related environment variables and their effects. (.github/copilot-instructions.md)

PR Type

Enhancement


Description

  • Replace Terser/CSS minifiers with esbuild for faster builds

  • Enable thread-loader by default with configurable parallelism

  • Add filesystem cache compression control and parallel build options

  • Switch to sass-embedded and style-loader for dev builds

  • Add GitHub Actions dependency caching for CI acceleration


Diagram Walkthrough

flowchart LR
  A["Old Build System"] --> B["esbuild Minification"]
  A --> C["Thread-loader Parallelism"]
  A --> D["Advanced Webpack Caching"]
  A --> E["sass-embedded Processing"]
  B --> F["Faster Production Builds"]
  C --> F
  D --> F
  E --> F
  G["GitHub Actions"] --> H["Dependency Caching"]
  H --> I["Faster CI Builds"]

File Walkthrough

Relevant files
Enhancement
build.mjs
Major build system optimization with esbuild and threading

build.mjs

  • Replace Terser with esbuild for JS/CSS minification
  • Add thread-loader with configurable worker count
  • Implement filesystem cache compression control
  • Enable parallel/sequential build modes via BUILD_PARALLEL
  • Switch to sass-embedded and conditional style-loader usage
  • Add watch-once mode for development builds
+167/-51
pre-release-build.yml
Add dependency and build caching to CI workflow                   

.github/workflows/pre-release-build.yml

  • Add npm cache using actions/cache@v4
  • Add Webpack filesystem cache for build acceleration
  • Cache both ~/.npm and .cache/webpack directories
+16/-0   
Dependencies
package.json
Update dependencies for build optimization                             

package.json

  • Add esbuild and esbuild-loader dependencies
  • Replace terser-webpack-plugin with esbuild-loader
  • Update sass to sass-embedded with newer sass-loader
  • Add style-loader and thread-loader dependencies
+7/-3     
Documentation
copilot-instructions.md
Document new build performance configuration options         

.github/copilot-instructions.md

  • Document new build performance environment variables
  • Explain BUILD_PARALLEL, BUILD_THREAD, BUILD_CACHE_COMPRESSION options
  • Add BUILD_WATCH_ONCE documentation for development
+16/-0   

Summary by CodeRabbit

  • Chores
    • Improved pre-release CI: Node-major-aware cache keys and enabled npm + Webpack caching to speed up builds.
  • Refactor
    • Switched to esbuild-based JS/CSS minification and updated Sass/CSS loader pipeline.
    • Added parallel, variant-aware builds with better caching, threading, and more robust build/finalization behavior.
  • Documentation
    • Added Build Performance Options documenting env vars for parallelism, threading, cache compression, watch-once, source maps, and output layout.

PeterDaveHello avatar Aug 30 '25 20:08 PeterDaveHello