[WIP] first updates to introduce sockets

[ibaryshnikov]

Reopening of https://github.com/CraneStation/wasi-common/pull/139

It's an attempt to implement two first networking methods:

• sock_socket, which creates a file descriptor and stores some information like domain, type, protocol
• sock_connect, which opens connection using fd obtained in sock_socket

[ibaryshnikov]

@sunfishcode what is a correct place to add new flags for rights? I'm getting

error[E0425]: cannot find value `__WASI_RIGHT_SOCK_CONNECT` in this scope
  --> crates/wasi-common/src/wasi.rs:42:7
   |
42 |     | __WASI_RIGHT_SOCK_CONNECT;
   |       ^^^^^^^^^^^^^^^^^^^^^^^^^ help: a constant with a similar name exists: `__WASI_RIGHT_SOCK_SHUTDOWN`

even after I added __WASI_RIGHT_SOCK_CONNECT const in both C headers and crates/wasi-c/src/wasm32.rs

[ibaryshnikov]

I'm stuck with writing a first test, any advice here? I run tests with

cargo test socket --features wasm_tests -- --nocapture

The test is in crates/wasi-common/wasi-misc-tests/src/bin/sock_socket.rs and the interface is added in https://github.com/CraneStation/rust-wasi/pull/11

Error: import sock_socket was not found in module wasi_unstable

thread 'wasi_misc_tests::sock_socket' panicked at 'assertion failed: `(left == right)`
  left: `1`,
 right: `0`: the test returned a termination value with a non-zero status code (1) which indicates a failure', src/libtest/lib.rs:337:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
test wasi_misc_tests::sock_socket ... FAILED

failures:

failures:
    wasi_misc_tests::sock_socket

[kubkon]

@sunfishcode what is a correct place to add new flags for rights? I'm getting

error[E0425]: cannot find value `__WASI_RIGHT_SOCK_CONNECT` in this scope
  --> crates/wasi-common/src/wasi.rs:42:7
   |
42 |     | __WASI_RIGHT_SOCK_CONNECT;
   |       ^^^^^^^^^^^^^^^^^^^^^^^^^ help: a constant with a similar name exists: `__WASI_RIGHT_SOCK_SHUTDOWN`

even after I added __WASI_RIGHT_SOCK_CONNECT const in both C headers and crates/wasi-c/src/wasm32.rs

Nowadays, the WASI consts, flags, etc. are generated procedurally from witx spec files. You can find them here: WebAssembly/WASI/phases/snapshot/witx. So to add a new const for dev/testing, we can go one of two ways: either add the const manually in every place it is required for dev/testing (in wasi-common/src/wasi.rs and the test itself), or submit a PR to WebAssembly/WASI/phases/ephemeral/witx which is the dev staging area.

[kubkon]

I'm stuck with writing a first test, any advice here? I run tests with

cargo test socket --features wasm_tests -- --nocapture

The test is in crates/wasi-common/wasi-misc-tests/src/bin/sock_socket.rs and the interface is added in CraneStation/rust-wasi#11

Error: import sock_socket was not found in module wasi_unstable

thread 'wasi_misc_tests::sock_socket' panicked at 'assertion failed: `(left == right)`
  left: `1`,
 right: `0`: the test returned a termination value with a non-zero status code (1) which indicates a failure', src/libtest/lib.rs:337:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
test wasi_misc_tests::sock_socket ... FAILED

failures:

failures:
    wasi_misc_tests::sock_socket

A couple of general pointers here. For development and debugging, I strongly suggest running the tests with trace logging enabled. This can be done by prepending RUST_LOG=wasi_common=trace to the test invocation you've posted, so:

RUST_LOG=wasi_common=trace cargo test socket --features wasm_tests

(NB the --nocapture can be omitted here AFAIK).

Back to the original problem though (apologies for the digression). The panic error we can see in the test output you've posted tells us that there is no sock_socket syscall in the wasi_unstable Wasm module. In general, in order to add a new syscall to wasi_unstable and use it within wasmtime you'll also need to tweak crates/wasi a little bit, since this is the crate where all the "gluing" between wasi-common and wasmtime actually happens. So, to make your test run, you'll need to add a syscall prototype to crates/wasi/syscalls.rs to look similar to:

pub unsafe extern "C" fn sock_socket(
    vmctx: *mut VMContext,
    sock_domain: i32,
    sock_type: wasi::__wasi_filetype_t,
    sock_protocol: i32,
    fd_out_ptr: wasi32::uintptr_t,
) -> wasi::__wasi_errno_t {
    trace!("sock_socket");
    // now extract the WasiCtx struct
    let wasi_ctx = ok_or_errno!(get_wasi_ctx(&mut *vmctx));
    // now extract the memory
    let memory = ok_or_errno!(get_memory(&mut *vmctx));
    hostcalls::sock_socket(wasi_ctx, memory, sock_domain, sock_type, sock_protocol, fd_out_ptr)
}

This is only part of the story as we still need to hook up this syscall to the Wasm instance, and this requires tweaking crates/wasi/instantiate.rs like so:

signature!(sock_shutdown);
signature!(sock_socket); // <-- this is the new line that's needed

And that should be you :-)

[ibaryshnikov]

@kubkon thank you for help, now tests work!

[ibaryshnikov]

Is there a good place for a design discussion? For what I found so far, wasmtime supports granting access to directories like --dir some_dir. Based on this it's possible to suggest granting access to a port like --port 8000. Going farther, we can make it possible to grant access to the whole network, for example 192.0.20.0, or to a single address in this network: 192.0.20.3. It may be possible to set up rights this way: we can connect to the address if we have access to the network. It's just a very rough draft, I'd like to know whether it's a right direction or not.

[sunfishcode]

@ibaryshnikov Yes, feel free to file new issues here for discussing design topics.

I do think something like --port makes sense. We can express blocks of addresses with CIDR notation, for example. I wrote up some ideas about manifests here, which includes some syntax for sockets which may provide some ideas here as well (though not with that exact syntax -- among other things, | is a reserved character in popular shells).

I should also mention, explicitly specifying directories with --dir= and --mapdir= isn't very convenient, and I expect the same will end up being true of --port= or similar things. However I think it makes sense to start with these explicit mechanisms for now, and we can talk about how to make this and other things more convenient as a separate step.

[github-actions[bot]]

Subscribe to Label Action

This issue or pull request has been labeled: "w", "a", "s", "i"

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

[github-actions[bot]]

Subscribe to Label Action

This issue or pull request has been labeled: "w", "a", "s", "i"

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

[github-actions[bot]]

Subscribe to Label Action

This issue or pull request has been labeled: "wasi", "wasi:impl"

Users Subscribed to "wasi"

• @kubkon

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

[sunfishcode]

This PR was part of a sockets effort in the context of WASI 0.1, which is no longer active. WASI 0.2 has sockets support and it's now supported in Wasmtime 17.