rust-Libp2p WASM Coordination Issue

[mriise]

This issue is an attempt to gather information relating to WASM support for rust-libp2p as well as serve as somewhat of a tracking / discussion issue. I am unsure about using the discussions features that github has for this sort of thing, so issue it is.

Please note that this will be a "living" issue meaning updates and changes will be added as work moves forward. Feel free to add things that I missed or would be important to consider.

Main List

Tentative Roadmap

This is my thoughts on what a roadmap for getting all the rust things to have WASM as a tier 1 target

• [ ] Browser WASM first, outside of the browser the use case becomes weaker to use WASM. WASI will change this and be a good platform to move away from browser-only WASM into the wide world of what WASM/I can do. Browsers have the biggest need now though.
• [ ] Make solid and easy to use examples of WASM libp2p running in the browser
• [ ] Document things where lacking, this issue is a big part of this
• [ ] Setup testground environments for browser testing (js libp2p would benefit from this too)
• [ ] Bench with JS lib and start the discussion about moving the JS lib to use rust-WASM internally (this will probably take a while)
  • Using WASM shouldn't kill the dev experience as opposed to pure JS
• [ ] Benching should reveal some perf issues and bottlenecks that can/should be tackled
• [ ] (Not libp2p exclusive) open discussion about giving the WASM treatment to other libs using things learned here
• [ ] Work on desktop/server side WASM/WASI, see ending for features that will likely block this for a while
  • At some point we should probably contribute to the spec, especially if there are features that are appealing for p2p things.

Things to consider / look into

• [ ] Wee Alloc vs rlsf as global allocator
• [ ] Best way to poll async rust things
• [ ] What examples to add
• [ ] What documentation to add
• [ ] Runtime feature detection (also relevant for optimizations) https://github.com/GoogleChromeLabs/wasm-feature-detect
• [ ] WASM error handling

Optimization things

• [ ] JS FFI boundary things
• [ ] Reduce heap allocations where reasonable
• [ ] Zero copy where possible
• [ ] Multithreading? (big maybe for this)

Ext.

Previous Implementations of WASM libp2p

• https://github.com/paritytech/smoldot
• https://github.com/iotaledger/identity.rs
• https://github.com/sireliah/wasm-p2p-experiment

Future WASM things

This is a list of potentially relevant WASM specs, most are in progress still

• Interface types
  • https://github.com/WebAssembly/interface-types/blob/main/proposals/interface-types/Explainer.md
• WASM Threads
  • https://github.com/GoogleChromeLabs/wasm-bindgen-rayon
  • https://github.com/wngr/wasm-futures-executor
  • https://github.com/WebAssembly/threads/blob/main/proposals/threads/Overview.md
• Host GC (instead of embedded allocator like wee-alloc)
  • https://github.com/WebAssembly/gc/blob/master/proposals/gc/Overview.md
• WASI
  • Networking (Sockets) https://github.com/WebAssembly/wasi-sockets
  • External GC (above)
  • access to large filesystem / persistence, wasm 64?

[mriise]

Pinging people that have previously worked on WASM here (that I know of) @wngr @tomaka @expenses

Also pinging @mxinden

Again, feel free to make suggestions or LMK if I missed anything that should be included!

[olanod]

A different kind of WASM support I'd like to bring to the table and something I'd like to experiment in the coming months(ideas and help will be welcomed) is the definition of a WASI interface that can be proposed later as wasi-p2p or something like that. Is a slightly different kind of WASM integration as the current direction is to be able to compile rust-libp2p to WASM expecting the host to provide calls that abstract the transport like using WebRTC with a set of custom host calls or perhaps the more standardized approach of targeting WASI that will have the sockets abstraction(could/will be polyfilled in the browser).

The proposal I mention is not so much for the browser support or environments that usually run a single bigger WASM blob where libp2p is included, my use case is embedded hardware where I want to allow a very constrained device to run a lightweight WASM interpreter like WASM3 and run many plugins that need to be as lightweight as possible, in systems where 1Mb is a lot and all there is I want to squeeze as many of those plugins as possible.
This all means working on no_std support of rust-libp2p to run natively on the microcontroller using whatever is available in the chip like crypto acceleration and different connectivity stacks(LORA, bluetooth, wifi+smoltcp, etc.) and then exposing libp2p to the WASM plugin as an interface so they don't have to bundle it that can be used directly or via a version of libp2p that is just a shell that makes access to the lower level "syscalls" more convenient.

[thomaseizinger]

A different kind of WASM support I'd like to bring to the table and something I'd like to experiment in the coming months(ideas and help will be welcomed) is the definition of a WASI interface that can be proposed later as wasi-p2p or something like that. Is a slightly different kind of WASM integration as the current direction is to be able to compile rust-libp2p to WASM expecting the host to provide calls that abstract the transport like using WebRTC with a set of custom host calls or perhaps the more standardized approach of targeting WASI that will have the sockets abstraction(could/will be polyfilled in the browser).

The proposal I mention is not so much for the browser support or environments that usually run a single bigger WASM blob where libp2p is included, my use case is embedded hardware where I want to allow a very constrained device to run a lightweight WASM interpreter like WASM3 and run many plugins that need to be as lightweight as possible, in systems where 1Mb is a lot and all there is I want to squeeze as many of those plugins as possible. This all means working on no_std support of rust-libp2p to run natively on the microcontroller using whatever is available in the chip like crypto acceleration and different connectivity stacks(LORA, bluetooth, wifi+smoltcp, etc.) and then exposing libp2p to the WASM plugin as an interface so they don't have to bundle it that can be used directly or via a version of libp2p that is just a shell that makes access to the lower level "syscalls" more convenient.

These are some very interesting and exciting ideas! Here are some thoughts on it:

rust-libp2p already implements a kind of plugin architecture. NetworkBehaviours are essentially plugins that can open / close new connections, open substreams etc. I am not sure if going for a wasi-p2p specification will be the most fruitful effort. What I could imagine to work much better is to try and get libp2p-swarm running on the micro controller natively with some static set of transports (bluetooth, tcp, etc) and have each of your plugins expose a component that corresponds to the NetworkBehaviour interface.

Another advantage of this approach is that with the NetworkBehaviour interface boundary, you are already getting a completely declarative interface around all the networking that is taking place. For example, you could easily log / intercept which component is establishing which network connections.

[vincev]

I have implemented a version of the chat example with libp2p WASM clients running in the browser connecting to a Floodsub server using a Websocket transport:

I added the steps to build and run it in the repo. To make deployment easier I have implemented a Websocket transport that uses Websys so that there is no need to package JS bindings and you can build a 100% rust client with a UI library like egui.

Keep in mind that I didn't do any serious testing, this is more of a proof of concept to test libp2p WASM support and get some feedback, I am happy to help and contribute any of this to libp2p if you think it may help.

[mxinden]

Wonderful work @vincev. Thanks for the elaborate README.md.

Keep in mind that I didn't do any serious testing, this is more of a proof of concept to test libp2p WASM support and get some feedback, I am happy to help and contribute any of this to libp2p if you think it may help.

Yes. That would be great! Off the top of my head I am thinking of moving a version of https://github.com/vincev/wasm-p2p-chat into misc/wasm-p2p-chat. What do you think? @thomaseizinger any comments?

[vincev]

Thanks @mxinden I am glad you like it.

Yes. That would be great! Off the top of my head I am thinking of moving a version of https://github.com/vincev/wasm-p2p-chat into misc/wasm-p2p-chat. What do you think? @thomaseizinger any comments?

Sounds good to me, another option would be to have an examples folder for WASM browser apps that would include examples for Websocket and WebCRT but I guess that can be done later.

[thomaseizinger]

Yes. That would be great! Off the top of my head I am thinking of moving a version of https://github.com/vincev/wasm-p2p-chat into misc/wasm-p2p-chat. What do you think? @thomaseizinger any comments?

Could be an opportunity to move towards https://github.com/libp2p/rust-libp2p/issues/3111.

[thomaseizinger]

@mriise Are you still interested in tracking these efforts? With https://github.com/libp2p/rust-libp2p/pull/4015, we now have a web-sys-based transport for WebTransport that is tested using the official interop-tests suite. We also have additional WASM-tests. Plus more and more protocol-crates now compile for WASM, see https://github.com/libp2p/rust-libp2p/pull/3973 for example.

More transports are being added, see https://github.com/libp2p/rust-libp2p/pull/4248 and https://github.com/libp2p/rust-libp2p/pull/4102.

I'd be curious what else we'd be missing. One thing I think would be really great is a full-stack rust-libp2p example where we have a server & a client, possibly reusing a common codebase and deploying them in the browser and as a server.

[DougAnderson444]

be really great is a full-stack rust-libp2p example where we have a server & a client, possibly reusing a common codebase and deploying them in the browser and as a server.

💯 Full stack is definitely appealing! I'm looking forward to being able to use protocols like gossipsub isomorphically in the browser and server, not having to trouble shoot or be familiar with 2 different implementations (Rust and Javascript) -- a real bonus.

[thomaseizinger]

Changelog:

I've updated the PR description to reflect the current state and deleted several of the old points which I found not to be actionable.