Loaders Team

Purpose

The Node.js Loaders Team maintains and actively develops the ECMAScript Modules Loaders implementation in Node.js core.

History

This team is spun off from the Modules team. We aim to implement the use cases that went unfulfilled by the initial ES modules implementation that can be achieved via loaders.

Project

• Resources

• Use cases

• Design

• Project board

• Meeting minutes

Status

Milestone 1: Parity with CommonJS

Before extending into new frontiers, we need to improve the loaders API enough that users can do just about everything they could do in CommonJS with ESM + loaders. (Outside of loaders scope, but related to the goal of parity between CommonJS and ESM, is finishing and stabilizing --experimental-vm-modules.)

• [x] Finish https://github.com/nodejs/node/pull/37468 / https://github.com/nodejs/node/pull/35524, simplifying the hooks to resolve, load and globalPreloadCode.

• [x] Refactor the internal Node ESMLoader hooks into resolve and load. Node’s internal loader already has no-ops for transformSource and getGlobalPreloadCode, so all this really entails is wrapping the internal getFormat and getSource with one function load (getFormat is used internally outside ESMLoader, so they cannot merely be merged). https://github.com/nodejs/node/pull/37468

• [x] Refactor Node’s internal ESM loader to move its exception on unknown file types from within resolve (on detection of unknown extensions) to within load (if the resolved extension has no defined translator). https://github.com/nodejs/node/pull/37468

• [x] Implement chaining as described in the design, where the default<hookName> becomes next and references the next registered hook in the chain. https://github.com/nodejs/node/pull/42623

• [ ] Have loaders apply to subsequent loaders. https://github.com/nodejs/loaders/blob/main/doc/design/proposal-ambient-loaders.md, https://github.com/nodejs/node/pull/43772

• [ ] Move loaders off thread. https://github.com/nodejs/node/issues/43658

  We hope that moving loaders off thread will allow us to preserve an async resolve hook while supporting the sync import.meta.resolve API. If that turns out to be unachievable, however, then:

  • [ ] Convert resolve from async to sync https://github.com/nodejs/node/pull/43363

    • [ ] Add an async resolve to module module

    • [ ] Consider an API for async operations before resolution begins, such as preImport https://github.com/nodejs/loaders/pull/89

Milestone 2: Usability improvements

• [ ] Provide a way to register loaders without a command-line flag, for example via a "loaders" field in package.json (#98 or https://github.com/nodejs/node/pull/43973).

• [ ] Support loading source when the return value of load has format: 'commonjs'. See https://github.com/nodejs/node/issues/34753#issuecomment-735921348 and https://github.com/nodejs/loaders-test/blob/835506a638c6002c1b2d42ab7137db3e7eda53fa/coffeescript-loader/loader.js#L45-L50.

• [ ] First-class support for import maps that doesn’t require a custom loader.

• [ ] Add helper/utility functions to reduce boilerplate in user-defined hooks.

  • [ ] Start with helpers for retrieving the closest parent package.json associated with a specifier string; and for retrieving the package.json for a particular package by name (which is not necessarily the same result).

  • [ ] Potentially include all the functions that make up the ESM resolution algorithm as defined in the spec. Create helper functions for each of the functions defined in that psuedocode: esmResolve, packageImportsResolve, packageResolve, esmFileFormat, packageSelfResolve, readPackageJson, packageExportsResolve, lookupPackageScope, packageTargetResolve, packageImportsExportsResolve, patternKeyCompare. (Not necessarily all with these exact names, but corresponding to these functions from the spec.)

  • [ ] Follow up with similar helper functions that make up what happens within Node’s internal load. (Definitions to come.)

• [ ] Helper/utility functions to allow access to the CommonJS named exports discovery algorithm (cjs-module-lexer).

• [ ] Hooks for customizing the REPL, including transpilation and tab completion. Support users pasting TypeScript (or CoffeeScript or whatever) into the REPL and having just as good an experience as with plain JavaScript.

  • [ ] Support top-level await in the REPL API, if possible.

• [ ] Hooks for customizing the stack trace (in other words, a hook version of Error.prepareStackTrace). This would allow transpiled languages to improve the output.

• [ ] Hooks for customizing filesystem calls, for allowing things like virtual filesystems or archives treated as volumes.

• [ ] Inherit configuration blob to worker threads and child processes.

• [ ] Provide a way for application code to communicate with loaders code.