wasi-common: add means to share handles across multiple modules

[ueno]

I am sure that this is not a good idea and there could be a better approach, but I would like to hear any opinions.

In our project, we want to open an I/O stream in a custom WASI module (or dynamically open it by the host) and somehow inject its handle to the main WASI module, to allow the guest to read/write with the fd_* functions. That is currently not possible because the FD management (EntryTable) is internal to the wasi-common crate.

I'm wondering if EntryTable can be made public, with a couple of registration/lookup methods, so other modules can register FD through it. A typical usage would be:

  // Create a centrally managed file descriptor table.
  let entries = EntryTable::new();

  // Create WasiCtx with the table.
  let mut builder = WasiCtxBuilder::new();
  let ctx = builder.entries(entries).build();

  // Create a context of an external WASI module with the same table.
  let mut builder = WasiOtherCtxBuilder::new();
  let ctx = builder.entries(entries).build();

The implementation of WasiOtherCtx can create a Handle internally and register it with EntryTable::insert_handle.

[github-actions[bot]]

Subscribe to Label Action

cc @kubkon

This issue or pull request has been labeled: "wasi"

Thus the following users have been cc'd because of the following labels:

• kubkon: wasi

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

[pchickey]

I agree there should be a facility to share handles across WasiCtxs. However, I'm not so sure that this is the correct approach.

The EntryTable is a map of capability handles (u32s) to capabilities (entries, which are always files at the moment, but will have lots of other types one day soon). This mapping needs to exist because wasm instances cannot yet use reference types, which can be used directly to implement capabilities. So, I think that it makes the most sense for an EntryTable to be used by only one single WasiCtx.

It is very desirable to share capabilities between different instances. I think we should expose the accessor, insertion, and deletion operations on a WasiCtx's entrytable as part of the WasiCtx api, so that the runtime can share a file between instances by getting it out of one wasictx and inserting it into another.

As an implementation note, I would prefer if entries were send and sync for multithreaded and async runtimes, so we'd want to use Arc instead of Rc, etc. A big part of my goals in #2205 is to get the scheduler abstraction modularized so it can be used in async rust, and I'll be adding support for async file read/write at some point soon as well.

[ueno]

Thank you for the suggestions, and sorry for taking long time to get back to this topic.

I think we should expose the accessor, insertion, and deletion operations on a WasiCtx's entrytable as part of the WasiCtx api, so that the runtime can share a file between instances by getting it out of one wasictx and inserting it into another.

I've modified the PR along those lines; would you check if this is acceptable? For testing purposes, I created a hypothetical memfd-like interface based on this.

[pchickey]

This looks exactly like what I was thinking, thanks.

Fwiw, since we had this conversation I embarked on a rewrite of wasi-common on top of https://github.com/bytecodealliance/system-interface that has a similar architecture for tables. I think it will end up being compatible with what we have here, but permit the wasi table to contain directories and other sort of handles/capabilities (sockets, http, etc), instead of treating Fd as both file and directories. https://github.com/bytecodealliance/wasmtime/tree/pch/wasi_common_cap_std/crates/wasi-c2

This work is pretty incomplete, at the moment it is focused on fleshing out system-interface and cap-std to have all the facilities required to implement wasi, so that we can abandon the gnarly wasi-common::sys hierarchy in favor of those crates. It will eventually be merged into the wasi-common crate, but I found it easier to start from a clean slate for this sort of exploration.

[ueno]

but permit the wasi table to contain directories and other sort of handles/capabilities (sockets, http, etc), instead of treating Fd as both file and directories.

I tried to port this PR to wasi-c2 and it seems to be indeed flexible; thank you for doing that! The usage example is also updated.

I forgot to mention that before, but my motivation (for Enarx) is to allow Wasm modules to communicate through a secure connection (e.g. TLS, QUIC, VSOCK) dynamically established after WasiCtx is instantiated. Therefore I would like to see a couple of things on top of it:

• an abstraction of bi-directional channels, as I commented on the wasi-c2 PR
• a way to override poll_oneoff behavior; I guess it would be possible through the io::ReadReady trait in system-interface

Here is a background discussion about these.

Having said that, is there any chance that this change to the current wasi-common will get merged, so the approach can be (partially) experimented before wasi-c2 becomes ready?

[pchickey]

Thanks for investigating how everything will work in the wasi-c2 paradigm. It is very validating to hear that it is working for you :).

Bi-directional channels should be implementable as a crate via impls of the WasiFile trait. There's no reason that the existing ReadPipe and WritePipe code lives in the wasi-c2 crate at the moment besides convenience. I'll probably move it out at some point to make the boundaries clearer. And indeed even the impl of the WasiFile and WasiDir traits in terms of cap-std::fs should be possible to do in an external crate. I may move this to a separate crate so that systems can make it clear that their Wasi engine cannot access their system's filesystem by not having cap-std::fs compiled in at all.

The remaining major goal I had when I started this rewrite was making the poll_oneoff behavior extensible by way of async Rust. My current plan is to land wasi-c2 (by deleting wasi-common and then renaming wasi-c2 to wasi-common) with only the existing synchronous (poll_oneoff implemented by select(2)) behavior, and work on the adaptation to optionally use async as a follow-up PR. The biggest stumbling block is that we don't want to make every wasi-common user also take a dep on tokio, so any async functionality we build will have to be opt-in.

[alexcrichton]

This is a pretty old PR at this point and lots has changed in the interim, especially around WASI, so I'm going to close this. If there's still bits to pull in today though I think it'd be good to update and re-send.