sdk-rust icon indicating copy to clipboard operation
sdk-rust copied to clipboard

WASI support

Open No9 opened this issue 2 years ago • 6 comments

Hi All, Opening this draft to solicit feedback and gauge interest. I've been building out a kn func project for WASI using WASMEdge https://github.com/uirlis/func-rust-wasi/tree/main/cloudevents

As part of that work I have put together a first pass at getting cloudevents to compile for WASI which is this PR.

The majority of the work is redefining the features in the Cargo.toml to bring in hyper-wasi when the OS is wasi

~~However I had to patch rust-claim in the test suite as the autocfg doesn't play well with a hyper-wasi so this placeholder is used https://github.com/cloudevents/sdk-rust/compare/main...No9:wasi-support?expand=1#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542R74. Which points to this patch https://github.com/svartalf/rust-claim/compare/master...No9:rust-claim:ignore-autocfg-wasi~~ No longer an issue as latest active claims repo has been added.

To build and run this PR you will need: WASMEdge installed - https://wasmedge.org/book/en/quick_start/install.html WASI Tools for Rust - cargo install cargo-wasi Set the was env - export CARGO_TARGET_WASM32_WASI_RUNNER=wasmedge

Currently this PR should pass testing with with the following command that enables the features used in the sample kn project

cargo test --features "http-binding hyper hyper_wasi" --target=wasm32-wasi
... 
test result: ok. 57 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

running 2 tests
test test_html_root_url ... ignored
test test_readme_deps ... ignored

test result: ok. 0 passed; 0 failed; 2 ignored; 0 measured; 0 filtered out 

And the main contribution command cargo test --all-features --all also passes

But the main contribution command with wasm32-wasi fails as expected with numerous errors.

cargo build --all-features --all --target=wasm32-wasi 

If this is of interest and we can work out what to do about claim I'm happy working on the CI pieces to finalise. Also understand if this is using projects that are too nascent and you want to keep it down stream for a while.

No9 avatar Jan 05 '23 17:01 No9

This is very cool. Thanks!

With "Wasm containers" supported in containerd and Docker Desktop etc, developers can build very small container apps to respond to events on demand -- image size for a Wasm app is often 1-2 MB as opposed to 20+MB for LXC; and Wasm starts up much faster than LXC apps.

I would like to understand if there is anything we can do on the hyper_wasi side to help with the issue with the autocfg / claim crates? Thanks.

juntao avatar Feb 09 '23 09:02 juntao

@juntao Thanks for the feedback. As you mention the main item is the autocfg changes as it was just a naive removal without to much consideration as I wanted to gauge interest here. I don't think there is anything that can be done in hyper-wasi as the problem is around target detection. There will also be a modification to the build system here to test the wasm path. @jcrossley3 @markmandel If we can resolve the dependency issues and build item would you know if there is interest in landing this? cc/ @font

No9 avatar Feb 09 '23 15:02 No9

Thanks @No9

I looked into the autocfg issue. It seems that it can be fixed with a patch section in cargo.toml file to force the autocfg dependency in hyper_wasi / tokio_wasi at 1.0 (the published crates have it on 1.1).

juntao avatar Feb 09 '23 17:02 juntao

@juntao Great news! Do you want to PR it in and we can monitor? claim is used quite widely so it would be great to get it in. Happy to do it but it won't be until next week. [Edit] Actually it looks like claim has died https://github.com/svartalf/rust-claim/issues/12 and has been picked up here https://github.com/mattwilkinsonn/rust-claims with the 1.1 patch. I'll validate it and add it to this PR.

No9 avatar Feb 09 '23 17:02 No9

@juntao That worked out and I've updated the PR. Once we get the workflows enabled I'll add the build config

No9 avatar Feb 09 '23 18:02 No9

Very cool. Thank you!

I look forward to building and deploying cloudevent functions in Wasm containers!

juntao avatar Feb 09 '23 18:02 juntao

@jcrossley3 Now that wasm support has been added to OpenShift in developer preview with Knative support my understanding is that this is based on wasmedge. https://www.redhat.com/en/blog/webassembly-wasm-and-openshift-a-powerful-duo-for-modern-applications

is there anychance we can get this PR to land? cc @gauravsingh85 @mrunalp @sohankunkerkar

No9 avatar Mar 17 '24 00:03 No9

@No9 Can you rebase through the conflicts, please?

jcrossley3 avatar Mar 18 '24 15:03 jcrossley3

@Lazzaretti @jcrossley3 I've rebased and merged the changes Clicking the accept changes in the GH UI broke the DCO so I reset and applied the tick changes by hand. The workflow will need to be updated to run the tests as only the WASI reqwest system is currently tested. Would you like that as part of this PR or should I open a separate one?

No9 avatar Apr 10 '24 08:04 No9

@No9 Thanks for the update! I would prefer to have the tests including CI in the same PR :)

Lazzaretti avatar Apr 10 '24 08:04 Lazzaretti

@Lazzaretti updated the PR to support wasm32-wasi target in the build system.

Notes: It installs wasmedge to run the tests as per these instructions here: https://wasmedge.org/docs/start/install/#install-for-all-users

nit: hyper had slipped out of the conditional in Cargo.toml during a merge so that has been fixed.

No9 avatar Apr 11 '24 21:04 No9

@No9 Thanks a lot for your contribution 🎉

Lazzaretti avatar Apr 30 '24 18:04 Lazzaretti