wasm-pack
wasm-pack copied to clipboard
`wasm-pack` should generate isomorphic code
💡 Feature description
wasm-pack
should generate isomorphic code
💻 Basic example
Below two targets have different output.
$ cd hello-world
$ wasm-pack build -t nodejs
$ wasm-pack build -t web
Differences are
-
node don't have init, while browser have init
-
node require
TextDecoder
from util, while browser check global -
node use
readFileSync
, while browser use fetch -
node generate CJS, while browser generate ESM
But,
-
can embed, instead of init
-
Quote https://nodejs.org/api/util.html,
The TextDecoder class is also available on the global object
-
And nodejs now support ESM for a long time. And now maintenance LTS is node16. Should drop CJS.
-
I think, this should be implement within
wasm-pack
, instead ofwasm-bindgen
So,
- I suggest,
wasm-pack
should embed the wasm generated bywasm-bindgen
as base64 string, and generate same ESM code for node and web
Related
-
emscripten support generate option
SINGLE_FILE
, which embed wasm as base64 string -
https://github.com/rustwasm/wasm-pack/issues/831
-
https://github.com/rustwasm/wasm-pack/issues/1253
-
https://github.com/rustwasm/wasm-pack/issues/1039
-
https://github.com/rustwasm/wasm-pack/issues/313
Please :pray:
Bump. This would be incredible to have. Is anyone on the team working on this? Can we support in anyway? 🫡
Isomorphic code would be amazing.
Workaround, tested same code works on both node and browser.
wasm-pack build --target nodejs --out-dir ./dist/pkg && node ./patch.mjs
import { readFile, writeFile } from "node:fs/promises";
const name = "xxx";
const content = await readFile(`./dist/pkg/${name}.js`, "utf8");
const patched = content
// use global TextDecoder TextEncoder
.replace("require(`util`)", "globalThis")
// inline bytes Uint8Array
.replace(
/\nconst path.*\nconst bytes.*\n/,
`
var __toBinary = /* @__PURE__ */ (() => {
var table = new Uint8Array(128);
for (var i = 0; i < 64; i++)
table[i < 26 ? i + 65 : i < 52 ? i + 71 : i < 62 ? i - 4 : i * 4 - 205] = i;
return (base64) => {
var n = base64.length, bytes = new Uint8Array((n - (base64[n - 1] == "=") - (base64[n - 2] == "=")) * 3 / 4 | 0);
for (var i2 = 0, j = 0; i2 < n; ) {
var c0 = table[base64.charCodeAt(i2++)], c1 = table[base64.charCodeAt(i2++)];
var c2 = table[base64.charCodeAt(i2++)], c3 = table[base64.charCodeAt(i2++)];
bytes[j++] = c0 << 2 | c1 >> 4;
bytes[j++] = c1 << 4 | c2 >> 2;
bytes[j++] = c2 << 6 | c3;
}
return bytes;
};
})();
const bytes = __toBinary(${JSON.stringify(await readFile(`./dist/pkg/${name}_bg.wasm`, "base64"))
});
`,
);
// deal with `imports['__wbindgen_placeholder__']`
// TODO: optimize with `__wbg_get_imports`
const wrapped = `export default (function() {
const module = { exports: {} };
${patched}
return module.exports;
})()
`;
await writeFile(`./dist/${name}.mjs`, wrapped);
@loynoir Awesome work! I love how you made it isomorphic. Previously, I was doing something similar, inlining base64 to bring wasm-pack libraries (web target) into a web worker within a SvelteKit project (because compiled SvelteKit web worker has difficulty in fetching wasm file unless inlined). I didn't think it could be extended this far, so kudos for that! 🤩
I have a quick question though. Your approach allows importing the entire wasm-pack lib like so:
import wasm from "my-wasm-lib";
const { add } = wasm;
console.log(add(1, 2))
Would it be feasible to import individual functions directly? Something akin to:
import { add } from "my-wasm-lib";
console.log(add(1, 2))
import { readFile, writeFile } from "node:fs/promises";
const cargoTomlContent = await readFile("./Cargo.toml", "utf8");
const cargoPackageName = /\[package\]\nname = "(.*?)"/.exec(cargoTomlContent)[1]
const name = cargoPackageName.replace(/-/g, '_')
const content = await readFile(`./dist/pkg/${name}.js`, "utf8");
const patched = content
// use global TextDecoder TextEncoder
.replace("require(`util`)", "globalThis")
// attach to `imports` instead of module.exports
.replace("= module.exports", "= imports")
.replace(/\nmodule\.exports\.(.*?)\s+/g, "\nexport const $1 = imports.$1 ")
.replace(/$/, 'export default imports')
// inline bytes Uint8Array
.replace(
/\nconst path.*\nconst bytes.*\n/,
`
var __toBinary = /* @__PURE__ */ (() => {
var table = new Uint8Array(128);
for (var i = 0; i < 64; i++)
table[i < 26 ? i + 65 : i < 52 ? i + 71 : i < 62 ? i - 4 : i * 4 - 205] = i;
return (base64) => {
var n = base64.length, bytes = new Uint8Array((n - (base64[n - 1] == "=") - (base64[n - 2] == "=")) * 3 / 4 | 0);
for (var i2 = 0, j = 0; i2 < n; ) {
var c0 = table[base64.charCodeAt(i2++)], c1 = table[base64.charCodeAt(i2++)];
var c2 = table[base64.charCodeAt(i2++)], c3 = table[base64.charCodeAt(i2++)];
bytes[j++] = c0 << 2 | c1 >> 4;
bytes[j++] = c1 << 4 | c2 >> 2;
bytes[j++] = c2 << 6 | c3;
}
return bytes;
};
})();
const bytes = __toBinary(${JSON.stringify(await readFile(`./dist/pkg/${name}_bg.wasm`, "base64"))
});
`,
);
await writeFile(`./dist/${name}.mjs`, patched);
@loynoir Works brilliantly! Thanks!
@loynoir Thanks for the solution, I'm using nuxt.js and I can use it in both server and client environments with just a little code modification
.replace("require('util')", "process.client?globalThis:require('util')")