bytecodealliance
Supporting a 'wasi:' prefix on imports
Right now, code that uses imports via WASI look like regular imports, with the package name being the first indicator:
import { someFn } from "ns:pkg/[email protected]";
export const run = {
run() { ... }
}
This works, but I think going forward life for embedders and tooling/ecosystem projects will be much easier if we prefix these imports with wasi: (or something similar) to note them/control how the imports are done.
The idea is that the above code would turn into:
import { someFn } from "wasi:ns:pkg/[email protected]";
export const run = {
run() { ... }
}
There are other ways to indicate the type of an import/control (i.e. import attributes, for example) , but prefixing with wasi: is incredibly simple and offers a point of leverage -- being able to easily tell which imports are "wasi"/underlying paltform imports. This mirrors the convention of node: as an import prefix which is used in more modern code bases (i.e. from "node:fs" rather than from "fs").
Changes may be required "below" and "above" componentize JS to make this work, but I figure that ComponentizeJS is a good place to discuss and eventually start this work.
Right now, code that uses imports via WASI look like regular imports
What do you mean by "imports via WASI"? If I understand what you're getting at correctly, then this isn't really about WASI, but about WIT / the Component Model? If so, then wasi: would definitely not be the right prefix, because that really is about WASI-specified APIs, not all Component Model imports.
Apart from that I'm also not necessarily entirely convinced that we even should have a solution like this. You mention the node: prefix convention, but that's about APIs built into Node.js, not any APIs loaded by Node.js that aren't implemented in JS. The more direct analog would be Node native modules loaded as shared libraries, and those aren't specifically denoted in source code—nor should they be.
I think it's actively a good thing to be able to implement APIs as CM imports transparently, without the consumer/importer having to know anything about it.
What do you mean by "imports via WASI"? If I understand what you're getting at correctly, then this isn't really about WASI, but about WIT / the Component Model? If so, then wasi: would definitely not be the right prefix, because that really is about WASI-specified APIs, not all Component Model imports.
Ah sorry yes -- I was thinking about WIT imports broadly but the WASI ones in particular. wit: works as well -- the core idea is to designate the modern wasm platform -- i.e. WIT-powered components via the Component Model.
Apart from that I'm also not necessarily entirely convinced that we even should have a solution like this. You mention the node: prefix convention, but that's about APIs built into Node.js, not any APIs loaded by Node.js that aren't implemented in JS. The more direct analog would be Node native modules loaded as shared libraries, and those aren't specifically denoted in source code—nor should they be.
Yeah so the node: prefix works to identify node as a the platform builtin that's being imported -- wit imports are platform provided "builtins" in a similar fashion -- i.e. they are not actually supplied by any local code.
Native modules are regular packages, from the point of the importer (the code that makes them work is locally available), so I'm not sure they are the right analog here.
I think it's actively a good thing to be able to implement APIs as CM imports transparently, without the consumer/importer having to know anything about it.
Hmnn I think I disagree here -- there's no such thing as a "transparent" import for the user, they have to know the WIT interface that they're importing, and they have to know what it means to do that import (any naive use of a bundler will fail, for example).
If you sue a WIT import these days (as we do), no bundler will know what to do with it, and will fail. This is fine for pure JS workflows, but there's not much difference between wasi:ns:... and ns:.... You can't make packages with those names anyway, they can't really resolve easily to regular files AFAIK, etc. There's basically nothing transparent or familiar about these strings as imports.
The point that "transparent" imports make sense is having regular packages export code that runs on the WebAssembly platform, but that's a completely different discussion -- that's more to do with Node's conditional export system and WinterTC runtime-keys. To illustrate, being able to import "axios" and have that resolve to a wasm conditional export as necessary depending on the platform/package resolution/bundler. That's a different problem w/ a different solution.
Hmm, I'm not sure I understand what you think this will solve.
I think going forward life for embedders and tooling/ecosystem projects will be much easier
Can you say in which specific ways it will make life easier for either embedders or tools?
Ah so the main improvement I think here will be making it easier to know which imports are WIT-related/are related to the component model, similar to how node:... makes it clear where the builtin in supposed to come from.
The primary benefit in the node ecosystem is avoiding name collisions (even when rare) but I know I generally write NodeJS code and use node: prefixes wherever I can to make it very clear what I'm intending to import. But a secondary benefit is that it makes it easier to write code for other JS environments (e.g. Deno's docs related to the node builtins)
Roughly, I'm trying to think ahead to what it would take to write a Rollup (and later Rolldown) plugin that interacts with/triggers for imports that are WIT-based/platform provided.
It's a simple question of detecting an import of ns:pkg/[email protected] versus wit:ns:pkg/[email protected] (for example), but having that prefix makes it possible for me to trivially write a rollup plugin that could do something like automatically generate the WIT file from just source code, generate the right types on the fly, etc. Obviously generating the export is much more difficult if we're not using Typescript but at the very least I can tell the difference between some import that is going to be filled in some other way, and one that should be filled in via a WIT import.
Another use case that I'm envisioning here is that if I know that an import is a WIT import, I can attempt to auto-fill/connect implementations (which could be JS packages from NPM or elsewhere). This isn't impossible to do without the prefix, it just makes detecting when it is warranted trivial.
This also enables us to create/template out configurations for bundlers, tsc and other tooling that can by default ignore wit:* imports.
The rationale here is the same as the import ... from "fs" versus import ... from "node:fs", etc. Knowing where the platform dependency is coming and being able to plugin helpful import semantics is useful.
That said, I could totally see an argument for import attributes here rather than the prefix, as the import attributes can pack more information. Given that point, I still think there is some benefit in the prefix as a middle ground and good distinguisher. Imports already look like they're prefixed to the average NodeJS dev -- the prefix is just the WIT namespace (but not intentionally).
The rationale here is the same as the
import ... from "fs"versusimport ... from "node:fs", etc. Knowing where the platform dependency is coming and being able to plugin helpful import semantics is useful.
This, to me, still isn't right. node: means "this module is available in the Node runtime". Whereas your wit: prefix would only mean "this module uses wit as the interfacing mechanism". I still think that that is much more immediately analogous to native modules, which don't have a specific prefix.
And as you point out, bundlers will already choke on WIT imports because of the ns:pkg syntax.
but having that prefix makes it possible for me to trivially write a rollup plugin that could do something like automatically generate the WIT file from just source code,
How would that work? An import doesn't contain any interface definitions, so I don't see how you could derive a WIT file from it. Doing that from a TypeScript implementation of a component is a different matter and indeed very doable, but also unrelated to importing.
At the end of the day I'm not extremely strongly opposed to a prefix, but I'm still pretty skeptical of the value proposition.
This, to me, still isn't right. node: means "this module is available in the Node runtime". Whereas your wit: prefix would only mean "this module uses wit as the interfacing mechanism". I still think that that is much more immediately analogous to native modules, which don't have a specific prefix.
And as you point out, bundlers will already choke on WIT imports because of the ns:pkg syntax.
Maybe we're over-indexing on the wit: as a prefix -- the point is to identify the import as a component model import, i.e. something that must be provided from the platform. I don't think the wit: prefix only means "this module uses wit as the interfacing mechanism", because WIT is intrinsically tied to the Component Model.
BUT, what it's analogous to is besides the point, I think.
Agree with you that bundlers already choke on WIT imports, and I'd like to be able to write something that easily handles all of them, unambiguously. This would make for a good add-this-to-your-bundler-setup plugin, or something other tooling could add for people.
How would that work? An import doesn't contain any interface definitions, so I don't see how you could derive a WIT file from it. Doing that from a TypeScript implementation of a component is a different matter and indeed very doable, but also unrelated to importing.
At the end of the day I'm not extremely strongly opposed to a prefix, but I'm still pretty skeptical of the value proposition.
I think this is outside scope of comopnentize-js of course, but tooling that knows where the interface is (or is configured properly) would be able to find the relevant WIT and pull it down. In a very simplistic sense, imagine a registry of known packages pre-configured at the tool level, and being able to pull the WITs appropriately.
The dumbest version of this could just gather the imports, put them in a WIT file, and then call wkg wit fetch (which will obviously fail on unknown WIT interfaces, but you get the point).
The relevance of the Typescript was more towards the use case of generating the entire top level wit. We know the imports from the the imports statements, and we actually know the exports from the exports statements, so I was thinking more along the lines of what would it take to generate the exports too -- we basically need to know the types involved, and if we can figure all those out, we could actually work "backwards" and generate the outgoing interfaces too (would probably be impossible to automatically determine if something was trying to export wasi:cli/run, but we at least know some interface named run with a run function is being exported.
Yeah, maybe this is the kind of thing that is much easier to motivate once there's an example/PoC which shows the potential/unlock.
Yeah, maybe this is the kind of thing that is much easier to motivate once there's an example/PoC which shows the potential/unlock.
No no, I totally get the idea of generating WIT from source code. I just think that nothing we do with import statements will ever help with that, because that's simply not the place the type information is available—nor the place where we would even want to generate WIT. For exports however, this absolutely makes sense, and is something I've long been thinking about.
To the overall thing, something I just remembered is that there are now reservations for the wasm: and wasm-js: protocols, which sets a precedent that seems useful here. With that, I think I'm warming up to the idea of wit:. (component: would be better in a world where that term wasn't overloaded by a few 100x.)
No no, I totally get the idea of generating WIT from source code. I just think that nothing we do with import statements will ever help with that, because that's simply not the place the type information is available—nor the place where we would even want to generate WIT. For exports however, this absolutely makes sense, and is something I've long been thinking about.
Yeah funnily enough I think the exports is the hard part and the imports is the easy part! I'm basically thinking that a properly configured
To the overall thing, something I just remembered is that there are now reservations for the wasm: and wasm-js: protocols, which sets a precedent that seems useful here. With that, I think I'm warming up to the idea of wit:. (component: would be better in a world where that term wasn't overloaded by a few 100x.)
Oh, I did not know this -- that's very helpful. I have no idea what this thing should be called, but I really just want a way to lazily know what is wasm-ecosystem related so we can do smart things that hopefully improve DX, feels like there is a lot there.
And yeah... we're at this weird point where the words are either insanely overloaded, or almost completely unfamiliar. wasm:* feels like "take from this component", which IS also a great valid usecase and example and could work a very similar way. wit: fits better with what I was hoping for, but your note makes me think that there is destined to exist somewhere a docs page that explains the difference (so maybe unifying/simplifying could be worth it early... does wasm:* work for everything...?)