API for embedding Spin from non-Rust programs

[radu-matei]

Currently, embedding the Spin execution context (or particular triggers) can only be done from Rust programs.

Do we want to have a C API that would allow to create embedding APIs for languages like Go or C#?

[itowlson]

Some not very organised thoughts on this:

• There are (at least) three "interesting" operations in Spin - listening, routing and handling.
  • Listening is a long-running operation that monitors for when events occur
  • Routing determines which Wasm module should handle an event, and with what settings (in the broadest sense, e.g. file mounts, env vars, etc.)
  • Handling runs the selected module with the selected settings.
• If you just want a specialised Spin build in your favourite language, then you want to import the whole shebang - essentially a spin_up entry point that tells Spin to listen, route and handle.
• There may, though, be interesting cases where you want to decouple these. Consider an application that wants to use the BEAM (Erlang) substrate for its process supervision and orchestration model, but wants to implement guest code in Wasm using Spin. Or you have years of operations experience with your favourite Web server and want to use that instead of the Spin listener.
  • The case where you want a cut-down Spin build for your constrained device may also be relevant here - you don't want to bring in all of hyper, you just want to use your OS's super lightweight HTTP API.
  • To be clear, I'm not sure how realistic the BEAM example is, because the supervision model seems more relevant if processes are relatively long-lived and interacting, which would not be the case for Wasm guests - and the bits that were long running, such as the listener, you'd want to implement in BEAM. I need to understand this better!
  • If this is a scenario, then we also need C entry points into the individual trigger crates, e.g. spin_process_http_event, spin_process_redis_event, etc., i.e. entering at the 'routing' stage.
  • I'm not sure whether there is a scenario for pure handling.
• Building Spin as a staticlib binary (a .a file) works okay, but the build seemed a bit fiddly to me, and I wasn't clear how it would work with non-C languages. The trouble was that cargo seemed not to embed the floating-point and SSL libraries, so these had to be linked at the C stage.
  • I've not yet tried building as a cdylib to see if that links things fully.
  • Alex Crichton has some basic examples of calling from other languages into Rust at https://github.com/alexcrichton/rust-ffi-examples but his Rust code is very simple and doesn't run into this!

[itowlson]

A possible handling scenario is presumably the plug-in kind of environment, where the host is the one raising events/commands/queries directly to the plug-in, and doesn't necessarily need routing.

[gedw99]

I think this is a +1 from me....

Any chance that Golang can be supported for this ?

I was just reading https://spin.fermyon.dev/extending-and-embedding/, because i was looking for the ability to run Spin locally, and have the WASI Modules ( written in golang / tinygo) pipe to each other over stdout, just like the HTTP and Redis ones do.

That way a developer can easy test things locally without a Server, and yet the server can use HTTP and Redis for IO between WASI Modules. It is also really good for edge topologies like IOT, where you want to run a light Spin and just use pipes locally to compose your "application". I dont know if Spin on embedded or Mobiles is a thing being thought about though.. Being Rust, i imagine there is a good chance its possible ?

I am also wondering if the data streaming over the pipe is able to be consumed from golang ? Not sure as i am a newbie to Rust.