wazero
wazero copied to clipboard
tetratelabs
Add tinygo and rust examples that include memory allocation
Hi, thanks for the awesome project! I was taking a look over examples, but I didn't find an example of passing complex values. I'm sorry if I missed something, but do you have something like this? https://wasmedge.org/book/en/embed/go/memory.html#pass-complex-parameters-to-wasm-functions
Thanks for the kind words: I'll help with this!
Meanwhile, while this isn't merged, there could be some good examples to harvest from here: https://github.com/summerwind/the-art-of-webassembly-go/pull/2
Regardless, I'll iterate on this as a priority and land examples toward that goal. Others have asked about string types, and I was hoping to do two-for-one as error handling usually implies that ex env.abort
I think I'll call the example tinygo as we already have it in our CI and the examples are particular to the host functions of the embedder. If you are not interested in tinygo, let me know!
I'm ok with TinyGo, but it would be great if you have Rust examples as well.
only issue with rust it it tends to generate huge wasm, and we'd have to check it in or require the rust toolchain to build the project: both are a bit heavy.
At point of api stability I was thinking we would end up using filter-branch to move the examples dir into a new project, but we aren't there, yet. In any case, maybe there's a way to have rust create non-huge wasm, just iirc it tends to be big even for small things.
This is all beside the point that a rust example is a good idea, just a matter of how we test it and right now our test target tests everything without any deps besides go.
TL;DR: will do a rust one also however that materializes!
So I decided that the most neutral example handoff seems to be using https://github.com/rustwasm. There are other sorts of libraries in various vendor repositories for rust and structured library bindings, and I think if we have an example with and without wasm-bindgen it shouldn't be too hard for developers to make their own choices on how to produce wasm with rust. In any case the point of an example from our perspective is giving a warm handoff and I think this will do it!
Thanks for considering it. Our project is written in Go, so it makes sense to start with TinyGo. Then, we will support Rust as well once wazero is ready. We'll take a look at examples in the repository you shared and try to integrate wazero into our project soon. Thanks.
oh heh thanks for clarifying, incidentally I misunderstood your priority on rust so we'll have a rust example anyway soon as I'm nearly done :D
wasm-bindgen gets pretty complicated quickly due to undocumented imports such as "wbindgen_externref_xform" "wbindgen_placeholder" and interactions with undocumented globals. I think using this is worse than showing direct memory allocation, so I'll switch back.
ok I managed to find and document the config to produce small wasm in rust. To finish the rust side needs a new function on memory to scan for a null character. Otherwise, it is a burden.. Rust is not the only one that uses null-terminated strings, so not a biggie. I'll finish this up tomorrow and then move to tinygo https://github.com/tetratelabs/wazero/pull/475
renamed one last time as I don't want to thrash the sourcecode with compiled binaries. The same example in tinygo and also rust will force a different thinking than each individually.
ps sorry about the delay, just we walked into a little bomb and I prioritized fixing the context first. Finishing the example is def on my "next" https://github.com/tetratelabs/wazero/pull/482
Due to things noticed I think these examples will be more broadly about allocation as that's the trickiest part. cc @danielpacak @neilalexander @efejjota @summerwind @hajimehoshi
What I plan to do is make a directory called allocation and inside that have separate directories for rust and tinygo (maybe one for manual with %.wat but not priority). inside allocation/README.md it can explain the general problems about allocation which each runtime has a slight different issue about. the subdirectories can then narrow the issue into how they handle allocation and also the tricky part about what signature types represent memory offsets (pointers), both in (params) and out (results).
Specifically, I think there isn't enough documentation on how these things work, at least not centralized in the language runtimes (or I've not found it yet). This code was unintuitive but works.. so I do think there's value in us formalizing approaches with pros and cons in examples here.
I looked carefully at the wasmedge examples originally cited and it seems they are doing a couple things that make things a little head-scratching
- they use Cstring in TinyGo even though it isn't normal to do this in either Go or TinyGo. Hence, there's some pointer searching going on to find zero. I think this is a forged example.
- Pointers in Rust need to also know the length, which is related to above, but not user friendly. It may be forged to try to make both examples use the same code as it makes one awkward and unrealistic.
- To show memory ownership transfer, they use undocumented exports in Go, and also assume there are no interleaved calls. I don't think it is safe to advertise this approach even if it leads to concise code as it could be a trap.
Meanwhile I opened another issue to clarify the undocumented "malloc" and "free" exports in tinygo https://github.com/tinygo-org/tinygo/issues/2788
TL;DR; I'm putting this on pause though hope the examples I put in earlier comments give a workaround meanwhile.
The current approach for TinyGo will break next release per https://github.com/tinygo-org/tinygo/pull/3142 I'll keep this open and close it with a new approach once merged.