Support Source Phase Imports

[guybedford]

Feature Use Case

The WebAssembly ESM Integration supports two modes of JavaScript imports - the direct instance-level ESM Integration allowing each Wasm module to have a singular instance state and imports to JS in turn, and a source phase import which allows obtaining the Wasm module in its uninstantiated but compiled state.

Source Phase Imports are at Stage 3 at TC39 and the ESM Integration Phase 3 of the WebAssembly proposal process.

The source phase import is a syntactical higher order import that shares the same module registry key in the underlying module system:

import source mod from './mod.wasm'; // source phase
import * as inst from './mod.wasm'; // instance phase

mod instanceof WebAssembly.Module; // true

In many ways for WebAssembly - the source phase imports form is actually usually recommended as custom instantiation and multiple instantiation is the typical norm.

The other benefit of using this form is better ergonomics and portability on the web (where it is currently being implemented) and Node.js (where it is supported under --experimental-wasm-modules).

Feature Proposal

For RollupJS the proposal for supporting this would be:

• Get SWC parser support
• Treat modules in the source phase as still running the resolver hook normally, without the phase affecting resolution (since it is resolution independent by definition)
• Also have modules in the source phase share the load hook and load key again without phase info
• Then support a new sourceTransform hook for source phase transforms of the load hook, or add a new option to the transform hook to indicate the source phase, where only at this stage of the pipline to we give unique identity to the final module for the source phase and instance phase as distinguished in this way
• From a bundling perspective, the source phase is effectively treated as having a unique final module identity
• By default the sourceTransform hook can throw in Rollup core, so that plugins like the Wasm plugin can handle providing the source phase implementation initially

Feedback very welcome, I will try to drum up support on the SWC first step in the mean time.

[guybedford]

Upstream SWC issue posted in https://github.com/swc-project/swc/issues/10843.

[lukastaegert]

Ok, trying to wrap my head around this. So as I understand this, a source import gives an abstract ModuleSource object that represents a previous form of the module. The easy part would be to have those as external imports and yes, in those cases we could simply use the same resolution logic. For actual bundling, here is what I understand so far

• The sourceTransform hook would be responsible to provide such a ModuleSource object
• It would receive the output of the load hook. Here I am not sure this is definitely what we want. After all, the load hook would usually return JavaScript, not just the untransformed source. But this could also be a peculiarity of Rollup

In general, the current WebAssembly handling in Rollup plugins would be just emit WebAssembly as files without any transformation. As such, they could either keep the source import or try to polyfill it. But I am not sure how the load hook would play into this, usually, the plugin would already return the instantiation code in the load hook, and not some binary WebAssembly or similar.

[guybedford]

Yes, it represents the module before it was linked against dependencies, in a form that could be linked against arbitrary dependencies.

The Wasm plugin today does actually return the buffer as a encoded string from the load hook, see https://github.com/rollup/plugins/blob/master/packages/wasm/src/index.ts#L78.

The theory here is that we want to encourage plugins to not consider the source phase as a separate representation but ideally it should share the same load hook so that it definitely does represent the same module identity.

I understand this might not fit the model in all cases, but the argument could always be ignored and a custom fetch performed.

Alternatively I'm happy to have the source phase hook just take a signature more like the load hook as well instead of like the transform hook.

The source transform would just return a new module with a default export that provides the source phase representation, allowing it to be effectively a new module. So from the bundle graph perspective it's just a new module. But from the user graph perspective and plugin system perspective, it's a higher order representation of the same module.

[sapphi-red]

The theory here is that we want to encourage plugins to not consider the source phase as a separate representation but ideally it should share the same load hook so that it definitely does represent the same module identity.

To encourage that, I think rollup would also need to encourage plugins to return the non-JS files as-is in the load hook, otherwise plugins won't be able to use the content returned by the load hook. I guess it's difficult to change the data-uri plugin to use transform hook because after loading the content, it's not possible to tell what the type of the content is.

I understand this might not fit the model in all cases, but the argument could always be ignored and a custom fetch performed.

It can be ignored, but it will still run the load hooks which could be costly.

[sapphi-red]

I wonder if this affects this.getModuleInfo and other APIs related to the module identity. If the imports of the content returned by load hook and the imports of the content returned by sourcePhase hook are different, I think ModuleInfo should be different for each phases.

maybe related https://github.com/rollup/rollup/issues/5694