Service Mesh WASM module use-case (ORIGINAL_DST socketopt and TLS support)

[keithmattix]

Hello! I'm an Istio maintainer intrigued at the possibility of a service mesh WASM module, and @pchickey pointed me to this repo so I could see the status of WASI preview 2. I came across #9 and read through the discussion, and I see that the getter/setter approach for specific sockopts was adopted in the spec. While I don't have the context to comment on that decision, I did want to provide the community with some more information on our use-cases from the service mesh side that will hopefully shed some light on the request.

In general, service meshes provide transparent proxy functionality, meaning that the application binary doesn't need to change anything about how it calls services to be onboarded to the mesh. A big part of how that "magic" works in a Kubernetes environment is via the SO_ORIGINAL_DST socket option that allows a proxy to retrieve the the original destination address on the socket after the ClusterIP Service does its load balancing.

Also absent from the current version of the spec is any kind of TLS support (probably for good reason!) which would be crucial for any kind of mesh module implementation. I'm aware of a few efforts to compile crypto modules to WASI, but the *.wit interfaces for handling/verifying certificates would need to be in the spec for a mesh to be feasible.

This is by no means a demand to modify the spec immediately; I simply wanted to provide context for what I believe to be a really interesting WASI use case, and I'd be happy to work with the community to try and make it happen at some point 😄

[badeend]

Hi, thank you for reaching out!

IIUC:

• you want to incorporate WASM into the Istio/Envoy sidecar itself, and your goal is not to WASM'ify the main application workload.
• Istio accepts unencrypted traffic from the application container and transparently encrypts that traffic before forwarding it to the outside world

Correct?

First; how do you intend to use WASM within Istio? Do you want to move the entire sidecar process (including listener sockets) into a single WASM module instance? Or do you want to use WASM more plugin/handler style, where (for example) each request gets its own WASM instance?

Second; Given that you're specifically asking about TLS support I guess you want to move the encryption into WASM too. For this I have no good answer. "TLS" is a complicated topic on its own, too big to "tack on" either wasi-sockets or wasi-crypto (both explicitly define TLS as a non-goal). In the true spirit of WASI, it's probably better off as its own module. But, as of yet, no one has made the effort to draft up a proposal.

[keithmattix]

Appreciate the comment! So a couple of things:

• I actually am looking to WASMify the application workload. I'm thinking about something akin to grpc's proxyless service mesh where the Istio control plane sends configuration to this hypothetical mesh module running alongside the main application. I'm looking to avoid Envoy and sidecars completely for a "native" WASM mesh experience
• This module would hook into the spec's socket calls (e.g. socket accept, create-tcp-socket, etc.) and do transparent proxying to a more specific address. So if the user wants to send traffic to a k8s Service, the module can intercept that, add TLS, load balance differently, apply policy, etc.

Ack on TLS...I definitely agree that it sits somewhere above sockets.

[badeend]

Hmm. In that case, I don't understand why you would need ORIGINAL_DST from within WASI at all.

Istio takes on the role of the application WASM runtime. The host implementation can do whatever it wants, including rerouting & encrypting traffic flows, without telling the application. There is no one saying that one WASI call must map 1:1 to an underlying systemcall. As matter of fact, virtualization is one of the core principles of WASI. Eg. your implementation of tcp::local-address may use ORIGINAL_DST, or read X-Forwarded-For, or use ProxyProtocol, or ...

The application running in WASM doesn't have to know about any of this. As far as it is concerned, it is talking directly to the endpoint without anything in between.

[badeend]

@keithmattix Was my answer of any help? Let me know if there's anything else you'd like discussing on this topic.

[keithmattix]

Oh yes it was! I've been doing research into virtualization and hope to have a response soon :)