wasm-bindgen icon indicating copy to clipboard operation
wasm-bindgen copied to clipboard
verified publisher icon rustwasm

Add wasm-bindgen-ld executable

Open RReverser opened this issue 4 months ago • 5 comments

Back in https://github.com/rust-lang/cargo/pull/3954 I've added target.*.runner = ... support to Cargo primarily to make it easier to run Wasm directly via cargo ... commands, without having to look for the produced artifacts in nested directories with a custom build tool.

That helped a lot with running tests, benchmarks and examples, but one area where a custom tool like wasm-pack is still needed is simply producing Wasm (+side assets like JS) for publishing.

Unfortunately, Cargo doesn't have post-build tasks despite a number of requests and RFCs over years, but as mentioned in a recent meeting, I believe we can work around that by piggy-backing on the existing target.*.linker option.

The idea is that we provide a custom executable wasm-bindgen-ld from the cli crate. That executable can be configured by the user in their .cargo/config.toml as target.wasm32-unknown-unknown.linker = "wasm-bindgen-ld".

When Rust invokes the configured linker, it passes arguments the following information (gathered with a temporary debug script set as a linker):

  • Current working directory: D:\wasm-bindgen\temp-project
  • Cargo environment variables:
    • CARGO_CRATE_NAME=temp_project
    • CARGO_HOME=D:\.cargo
    • CARGO_MAKEFLAGS=-j --jobserver-fds=__rust_jobserver_semaphore_3522682985 --jobserver-auth=__rust_jobserver_semaphore_3522682985
    • CARGO_MANIFEST_DIR=D:\wasm-bindgen\temp-project
    • CARGO_MANIFEST_PATH=D:\wasm-bindgen\temp-project\Cargo.toml
    • CARGO_PKG_AUTHORS=
    • CARGO_PKG_DESCRIPTION=
    • CARGO_PKG_HOMEPAGE=
    • CARGO_PKG_LICENSE=
    • CARGO_PKG_LICENSE_FILE=
    • CARGO_PKG_NAME=temp-project
    • CARGO_PKG_README=
    • CARGO_PKG_REPOSITORY=
    • CARGO_PKG_RUST_VERSION=
    • CARGO_PKG_VERSION=0.1.0
    • CARGO_PKG_VERSION_MAJOR=0
    • CARGO_PKG_VERSION_MINOR=1
    • CARGO_PKG_VERSION_PATCH=0
    • CARGO_PKG_VERSION_PRE=
    • CARGO_PRIMARY_PACKAGE=1
    • CARGO_SBOM_PATH=
  • Arguments passed to the linker:
    • --export=__heap_base
    • --export=__data_end
    • -z
    • stack-size=1048576
    • --stack-first
    • --allow-undefined
    • --no-demangle
    • --no-entry
    • D:\.cargo\target\wasm32-unknown-unknown\debug\deps\temp_project.18y8u09d9b0o0ob9uqucwicbl.0toelm1.rcgu.o
    • D:\.cargo\target\wasm32-unknown-unknown\debug\deps\temp_project.bjm3cixlp3p23ygsb8r63uetw.0toelm1.rcgu.o
    • C:\Users\me\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\wasm32-unknown-unknown\lib\libstd-b50d6517e0fd9463.rlib
    • C:\Users\me\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\wasm32-unknown-unknown\lib\libpanic_abort-10df544347be6f06.rlib
    • C:\Users\me\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\wasm32-unknown-unknown\lib\libdlmalloc-0ff646270b660f0a.rlib
    • C:\Users\me\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\wasm32-unknown-unknown\lib\librustc_demangle-b89aa78e8d1a0461.rlib
    • C:\Users\me\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\wasm32-unknown-unknown\lib\libstd_detect-149f9fcf5d4e3de0.rlib
    • C:\Users\me\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\wasm32-unknown-unknown\lib\libhashbrown-6eb30aa37dd4c431.rlib
    • C:\Users\me\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\wasm32-unknown-unknown\lib\librustc_std_workspace_alloc-13fa5e853fc1ceb2.rlib
    • C:\Users\me\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\wasm32-unknown-unknown\lib\libminiz_oxide-5fd9c9f88c5cf6ae.rlib
    • C:\Users\me\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\wasm32-unknown-unknown\lib\libadler2-733aa475a769dd36.rlib
    • C:\Users\me\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\wasm32-unknown-unknown\lib\libunwind-ea58b373f3cba01e.rlib
    • C:\Users\me\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\wasm32-unknown-unknown\lib\libcfg_if-1287c5ddfb68988f.rlib
    • C:\Users\me\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\wasm32-unknown-unknown\lib\liblibc-5ccf685ebfa7d8a6.rlib
    • C:\Users\me\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\wasm32-unknown-unknown\lib\librustc_std_workspace_core-e0b01b6f65e0bb7c.rlib
    • C:\Users\me\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\wasm32-unknown-unknown\lib\liballoc-a33843bc0a74256f.rlib
    • C:\Users\me\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\wasm32-unknown-unknown\lib\libcore-413703d31ea26aba.rlib
    • C:\Users\me\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\wasm32-unknown-unknown\lib\libcompiler_builtins-f1fb81b72d646848.rlib
    • -L
    • C:\Users\me\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\wasm32-unknown-unknown\lib\self-contained
    • -o
    • D:\.cargo\target\wasm32-unknown-unknown\debug\deps\temp_project.wasm
    • --gc-sections
    • --no-entry
    • -O0

Out of all this stuff, we only care about -o argument to know where the linker will put the raw Wasm file, and potentially CARGO_MANIFEST_DIR to know where we want to put the pkg subfolder by default (if we want to match wasm-pack behaviour and not wasm-bindgen's own where --out-dir is a mandatory argument).

All arguments and environment need to be passed by wasm-bindgen-ld to the wasm-ld subprocess, and then we can simply execute wasm-bindgen's own postprocessing on the produced Wasm file and generate the final Wasm+JS.

The only issue I'm concerned about is that we need to distinguish the main library vs examples and tests, as we don't want to produce the main JS+Wasm output when linking those.

While for examples Rust passes a clear path like -o D:\.cargo\target\wasm32-unknown-unknown\debug\examples\demo-9a0b809269285348.wasm and sets CARGO_BIN_NAME=demo, for integration tests Rust passes -o D:\.cargo\target\wasm32-unknown-unknown\debug\deps\foo-fa13eb71b0a929d8.wasm where the only distinguishing characteristic is the hash in the filename, but everything else including environment variables are the same as for building the actual library.

I suspect this will require additionally invoking cargo metadata to determine what we're building.

RReverser avatar Sep 05 '25 16:09 RReverser

While for examples Rust passes a clear path like -o D:\.cargo\target\wasm32-unknown-unknown\debug\examples\demo-9a0b809269285348.wasm and sets CARGO_BIN_NAME=demo, for integration tests Rust passes -o D:\.cargo\target\wasm32-unknown-unknown\debug\deps\foo-fa13eb71b0a929d8.wasm where the only distinguishing characteristic is the hash in the filename, but everything else including environment variables are the same as for building the actual library.

Looking at the resolution for https://github.com/rust-lang/cargo/issues/11689, we could probably make a similar case for integration tests and add CARGO_BIN_NAME for those to Cargo too, in which case the problem would go away.

RReverser avatar Sep 05 '25 16:09 RReverser

The only issue I'm concerned about is that we need to distinguish the main library vs examples and tests, as we don't want to produce the main JS+Wasm output when linking those.

Actually, maybe I'm overthinking it. Since we want cargo build to "just work", we could instead just follow Cargo convention rather than wasm-pack/wasm-bindgen convention and generate JS+final Wasm in the actual target directory.

This wouldn't hurt either examples or tests, as it would simply mean that preprocessing part of them is done and the configured target runner just needs to execute the generated .js.

RReverser avatar Sep 06 '25 11:09 RReverser

I'm strongly in favor of just following the Cargo convention. Cargo can figure out how to solve this specific problem.

We should probably follow https://github.com/rust-lang/cargo/issues/6790 and see if this works for us or otherwise provide feedback.

daxpedda avatar Sep 06 '25 16:09 daxpedda

We should probably follow rust-lang/cargo#6790 and see if this works for us or otherwise provide feedback.

Oh yeah that's a nice one.

Sgtm, so first step is definitely just do whatever Cargo does and produce the .js file in the same path and with the same basename as the .wasm supplied to the linker.

This would also match what Emscripten target is already doing.

RReverser avatar Sep 06 '25 18:09 RReverser

Extra context: although initially I thought of this separately, I recently found out this is exactly what wasip2 is doing via https://github.com/bytecodealliance/wasm-component-ld, so it's not such a crazy idea after all.

RReverser avatar Sep 23 '25 20:09 RReverser