Relationship to asset references proposal

[justinfagnani]

This seems like it could possibly subsume the asset references proposal: https://github.com/tc39/proposal-asset-references

Instead of a new keyword as in asset references:

asset Logo from "./logo.gif";

We could have a asset-reference type:

import Logo from "./logo.gif" as 'asset-reference';

cc @sebmarkbage

[sebmarkbage]

Technically either this or assert syntax could be repurposed for this syntax, but they’re not quite the same concept. It’s more of a hack.

These other has some interesting consequences on asset references. For example, when you do materialize the asset references do you have to provide the “as” and “assert” again?

An important consideration of asset references is that they should be able to express their eventual type.

Eg I should be able to define a function F<T>(Reference<T>): T and have the type be resolved from the module implicitly like other imports.

For that to be possible, you need to be able to know at the call site what as/assert will be used.

So I think asset references need to also be able to include “as” and “assert” in the syntax.

asset Foo from “foo” assert {} as “wasm-module”;

const M = import(Foo);

So I don’t think either of these syntax subsumes the need for a special asset syntax because all of them have to combine with asset references.

[lucacasonato]

I agree with @sebmarkbage's thoughts here. How would get an asset reference for a WASM module? Something like the below seems very strange to me.

import module from "./foo.wasm" as "asset-reference:wasm-module";

The semantics of what an asset reference vs a "regular import" makes it pretty clear that these should be distinct: an asset reference fetches the underlying module on use (when it is passed to import()), where as import reflection attributes do not change when a module is fetched, but rather how it is exposed to the client.

From my understanding asset references are in their most basic form a static form of new URL(specifier, import.meta.url) with attached import assertions (I'm sorry for the egregious oversimplification). It allows for creating static references to assets that can be passed around and loaded at a later time.

TLDR: import assertion attributes change how the loaded asset is represented in JS, and asset references change when a module is loaded.

---

Regarding having import reflections inside of asset references: I don't think I agree with this. Asset references reference an asset (opaque, in the case of WASM it is WASM module). Import reflections just change how that asset is represented for the JS API.

A single asset can be loaded with different reflections. The code below would only fetch and parse the https://example.com/fibonacci.wasm module once:

import { fib } from "https://example.com/fibonacci.wasm";
import module from "https://example.com/fibonacci.wasm" as "wasm-module";

With asset references being a reference to the asset, not the module, the asset reference would not yet contain the reflection. The reflection would only be applied when actually performing the import:

asset fibonacci_wasm from "https://example.com/fibonacci.wasm";

const { fib } = await import(fibonacci_wasm);
const module = await import(fibonacci_wasm, { as: "wasm-module" });

I have multiple reasons for this:

a) the import reflection is something inherent to JS imports. It is not relevant when passing an asset reference to some other API like URL.createObjectURL(). b) asset references reference assets, not modules, or module bindings c) as a user that passes an asset reference to some resource loader, I don't care or want to care what reflection of the module the resource loader ultimately uses to give me my desired output. I just give it a reference to the asset (a specifier in rough terms), and it then deals with loading that asset in the right format.

[sebmarkbage]

@lucacasonato I see. I buy that. Import reflections should be applied late, but import assertions would be applied early when combined with asset references.

The important thing is that then something like TypeScript needs to be able to encode the type for all possible import reflections. Otherwise you can't implement a generic version of a method that does that loading.

asset Foo from "module";
const foo: AParticularModuleInterface = instantiate(Foo);

This should be able to function instantiate<T>(reference: Reference<T>): Module<T> in a way that ensures that it correctly corresponds to the protocol.

That in turn puts certain constraints on what as can possibly be used for. I'd argue that the syntax is too generic for how it can be used then.

However, if that's satisfied, I retract my concern because as long as dynamic import(...) supports as on top of a reference you could still combine them.

Really I think the import reflection proposal's motivation is actually very close to asset references. Basically it wants to preload something without instantiating it so it's just an intermediate representation of it.

I've similarly wanted a reference to a JS module where I can synchronously execute its module body when needed instead of having that be done eagerly.

[ljharb]

Can't you do that with an intermediate module (even a data URI) that does:

export default () => import('specifier')

?

[sebmarkbage]

Which part? The main things asset references provide is just formalizing the import('specifier') and new URL('specifier', import.meta.url) patterns into a standard syntax and make it nicer by avoiding intermediates.

However, there are a bunch of other features that we use in production environments that are not covered by ESM which makes it a partial story.

• A smart Reference should contain a graph of modules so you can start fetching all of them without a waterfall.
• Preloading all of them over the network without executing the module body graph.
• Synchronous execution of the module body on demand. I.e. lazy execution without an async task/tick.
• Serializing the Reference over the wire.

The way we use our references is we don't only use it with the built-in import(...) but also with user space bundler features. All of these other features should probably be added to the language as well. Once you do, it's not equivalent anymore.

[ljharb]

Ah, i was responding to the last sentence in your comment about deferred execution.

[sebmarkbage]

Yea, the main thing is that import(...) is async. Not even a micro-task. Even if you preload into the HTTP cache, you need a macro-task to pull it up.

In React at least, if you render a tree of React.lazy components, that means that we have to unpop the stack and yield to the browser every time we hit one of these. Even if they're preloaded. That adds a ton of overhead. So the more we make things lazy, the worse the performance gets.

We work around this by using Webpack specific tricks to load all modules over the network into memory (lazy module initializer closures) and then call them synchronously on-demand.

[sebmarkbage]

What I really want is preload('specifier').then(module => execute(module).thisIsSynchronouslyAvailable);

[Jamesernator]

I've similarly wanted a reference to a JS module where I can synchronously execute its module body when needed instead of having that be done eagerly.

What I really want is preload('specifier').then(module => execute(module).thisIsSynchronouslyAvailable);

There is already a separate proposal for lazily evaluating module bodies which seems to be what you're primarily wanting out of asset references.

And I don't see that asset references by themselves really address any of the points as to how such lazy evaluated modules would work, it's purely just a syntax for declaring one exists.

---

Regarding a different point:

• A smart Reference should contain a graph of modules so you can start fetching all of them without a waterfall.

Like this is just what modulepreload (and preload) is for right? If you know what modules are depended on by something you can just send the appropriate Link: rel=modulepreload headers with that resource to avoid a waterfall.

Although there certainly could be an API for actually querying the status of a (module)preload as currently there doesn't seem to be a way to query the current state of preloads. Knowing the availability state of resources fetching/parsing would be a useful API even outside of the JS modules uses.

It might still be worth baking some of that directly into the JS spec though as it would allow you to detect network errors/parse errors long prior to attempting to evaluate. i.e. we could have something like:

await import.preload("some-specifier"); // This could still throw a parse/network error potentially
// Module is ready to evaluate whenever, actually importing the module can't
// fail with parse/network error anymore though

This would also fix your problem with HTTP cache load being a macrotask, as we could have import.preload even perform all linking and such (perhaps even call it import.link).

---

The important thing is that then something like TypeScript needs to be able to encode the type for all possible import reflections.

Because, as has been pointed out by others, setting as: "some-type" is likely to be done later anyway couldn't TypeScript use import Asset from "asset-specifier" as "asset"; just fine? Like TypeScript would just resolve "asset-specifier" as usual, and set the type of Asset to AssetReference<ModuleInfoForSpecifier>.

As a concrete example, suppose we had:

import WasmAsset from "./wasm-module.wasm" as "asset" assert { type: "webassembly" };

where ./wasm-module.wasm was detected to have single export compute : i32 -> i32 and an import log: (i32, i32) -> i32. Then TypeScript would set the type of WasmAsset to be AssetReference<WebAssemblyModule<{ imports: { log: (a: number, b: number) => number }, exports: { compute: (x: number) => number } }>> where WebAssemblyModule is some purely container type that allows types to work.

Userland code could easily use these types with TypeScript's rather powerful generics like:

async function instantiateModule<Imports, Exports>(
    ref: AssetReference<WebAssemblyModuleReference<{ imports: Imports, exports: Exports }>>,
    imports: Imports,
): Promise<WebAssembly.Instance<{ imports: Imports, exports: Exports }> {
    const module = await import(ref, { as: "wasm-module" }) as WebAssembly.Module<{ imports: Imports, exports: Exports }>;
    const instance = await WebAssembly.instantiate(module, imports) as WebAssembly.Instance<{ imports: Imports, exports: Exports }>;
}

And in practice this function would be unneccessary, as TypeScript could just type import() and WebAssembly.instantiate to have those types as neccessary i.e.:

interface ImportType {
    <Imports, Exports>(
        ref: AssetReference<WebAssemblyModuleReference<{ exports: Exports, imports: Imports }>,
        importOptions: { as: "wasm-module" },
    ): WebAssembly.Module<{ exports: Exports, imports: Imports }>;
}

interface WebAssembly {
    instantiate(
        module: BufferLike, 
        imports: Record<string, any>,
    ): WebAssembly.Instance<{ imports: Record<any, , exports: any }>;
    // This would be new with asset references being well typed
    instantiate<Imports, Exports>(
        module: WebAssembly.Module<{ imports: Imports, exports: Exports }>, 
        imports: Imports,
    ): WebAssembly.Instance<{ imports: Imports, exports: Exports }>;
}

[guybedford]

@sebmarkbage you mention the preloading use case as a goal of the asset references proposal:

What I really want is preload('specifier').then(module => execute(module).thisIsSynchronouslyAvailable);

This may be better achieved through a dedicated module preloading specification, something like import.preload:

await import.preload('specifier');
import('specifier');

Such a function could be specified such that the dynamic import would execute synchronously if the preload has completed, at least until top-level await is involved.

I think this may be better to approach as a separate specification to both asset references and import reflection - a dedicated module preloading specification. I'd be happy to collaborate on such a spec if there is interest in it.

[sebmarkbage]

Preloading locally is just one. Another use case is getting a reference on the server using server-side JS and then passing that reference along to trace where it's used and then use that as input to send an instruction to the client to preload.

The tricky part with addressing each part individually is that it's a long list of use cases. If there's no first-class concept of passing a reference around you can't build abstractions for those use cases programmatically in user space code.

The goal is also to do it with as little syntax as possible so that a local file can do it. This is not just for rare advanced cases but for very common cases.

E.g.

asset logo from "./logo.gif";
...
<img src={logo} />

This can ensure that the image is preloaded using early hints from the server, rendered as HTML with the correct URL, loaded and possibly preloaded as need client-side by the framework with hints it has.

Doing the same thing with one syntax for each feature would be way too much work for every image.

[guybedford]

@sebmarkbage to clarify, both Luca and I actually support the principle of the asset references proposal even as a possible import reflection as described in this issue. The reference use case is the one I'm more familiar with and definitely agree with regards to static benefits for bundlers etc.

On the other hand would it be possible to obtain an asset reference for a resource that has not yet already been loaded? In this case, having a higher level preloader which can take the asset reference as an input may make more sense.

My point is more that preloading mechanics may be better separated from asset references as higher-level techniques on top of asset references.

[Jack-Works]

I'm wondering if this can help TypeScript users.

Nowadays if you use an asset like this:

src/index.ts

const u = new URL('./x.png', import.meta.url)
await readFile(u)

And the file compiled into dist, will become a runtime error because x.png is not "compiled" or "copied" to the dist folder.

If we have this, maybe TypeScript knows it should also copy the asset files to the build target (but that depends on the TypeScript team, because theoretically they can also analyze new URL(..., import.meta.url)).

import u from './x.png' as 'asset-reference'
await readFile(u) // now OK because x.png is copied!