Add Portforward Support

[wlan0]

This PR is a working implementation of PortForwarding using this proposed containerd API: https://github.com/dmcgowan/containerd/commit/5b3c33590712875b9ed17bb269525e4c837719c6. While this PR cannot be merged until a new release of containerd is made with the linked commits, I wanted to make this PR to get early feedback on design and implementation.

It is quite a large PR, so I've summarized the most important parts below:

[containerd-shim-kata-v2]

• In order to support the new contaierd API, the kata shim must be started using a shimManager rather than by calling shimapi.Run
• The manager allows multiple plugins to be registered for containerd to call. In this case, we register TaskAPI plugin and PortForwardAPI plugin
• To respond to calls from containerd to PortforwardAPI plugin, a new PortForward API is added to virtcontainers/sandbox which takes in 3 params:
  • containerID
  • containerPort
  • dialURL
• containerd expects the shim to dial back to the provided dialURL and use that connection to bidirectionally transmit data between the connection and the container:port
• This implementation dials the given dialURL and simultaneously exposes a VSock listener for kata-agent to callback before calling kata-agent
• kata-agent is expected to connect to the vsock:port and perform a bidirectional copy between container:port and the vsock:port.
• On the sandbox side, the previously obtained connection to dialURL and the connection obtained from kata-agent dialing back are used as two endpoints to copy data bidirectionally between one another.
• Essentially, there are two stages of bidirectional copies
  • containerd <-> shim
  • shim <-> agent
• This implementation also includes vsock listener pooling to reuse listeners for multiple requests to the same container:port

[kata-agent]

• Implements PortForward Service that dials back to the given VSock and bidirectionally copies between container:port and vsock:port

Architecture

                              +-------------------------------------------------------+          + ----------------------+
                              | host                                                  +          + VM                    +
                              |                                                       +          +                       +
                              |                                                       +          +                       +
                              |                                                       +          +                       +
                              |    containerd                containerd-shim-kata-v2  +          +    kata-agent         +
               PortForward()  |    +------------------+      +-------------------+    +   vsock  +    +---------------+  +
client stream +------------------> | shim.PortForward +----->+ agent.PortForward +------------------> | PortForward   |  +
                              |    +------------------+      +-------------------+    +          +    +---------------+  +
                              +-------------------------------------------------------+          +-----------------------+

APIs

Containerd PortForward API

Message PortForwardRequest {
 	// The container id.
 	string id = 1;
 	// The port to be forwarded inside the container.
 	int32 port = 2;
 	// A Unix domain socket address
 	string addr = 3;
}

Virtcontainers/Sandbox PortForward API

PortForward(ctx context.Context, hostAddr string, containerID string, port uint32) error

Kata-agent PortForward API

Message PortForwardRequest {
 	// The container id.
	string container_id = 1;
 	// The port to be forwarded inside the container.	
	uint32 port = 2;
	// A VSock callback address
	uint32 vsock_port = 3;
}

TODO

• [ ] Add unit and integration tests

[katacontainersbot]

Can one of the admins verify this patch?

[wlan0]

@egernst PTAL

[egernst]

Is it feasible to break the PR into smaller commits to cover various parts of it? Example:

• updated proto
• agent changes to implement the proto
• host side changes to call the new proto
• vc/sandbox port forward API introduction
• pluggable service change
• introduction of new service?

Thanks Sid! 🚀

[wlan0]

Is it feasible to break the PR into smaller commits to cover various parts of it?

Done!

[egernst]

/test

[egernst]

/test

[stevefan1999-personal]

Wonder if this is still alive because I was running some obstacle trying to test services that requires port-forwarding, which is what Coder needs.

[milantracy]

I would like to know whether or not the draft is still valid? Or is there any other PR that has been merged?

p.s I am working on gVisor's portforward, I am wondering if there is anything I could help with or should be aware of :)

[wlan0]

The PR is still valid. Only reason we couldn't merge was because we were waiting for containerd-1.6, which provides the APIs needed to implement portforward (https://github.com/dmcgowan/containerd/commit/5b3c33590712875b9ed17bb269525e4c837719c6)

I haven't followed up since. If you're interested in taking this PR and pushing it to master, I'd be happy to support it

[milantracy]

thanks. I will follow up with the containerd's PR later since it has been inactive for a while.

btw, I am wondering what else that still needs to be completed for this PR. I will be happy to help.

[stevefan1999-personal]

@milantracy If you need help on this please let me know. I also need that upstream containerd PR fixed so that my project can also go on.

[milantracy]

just fyi, I recently start working on port forward interface in containerd based on https://github.com/dmcgowan/containerd/pull/1

I would like to have ppl from kata container community as reviewer(s), please let me know if you are interested

@egernst @wlan0 @stevefan1999-personal

[github-actions[bot]]

This PR has been opened without with no activity for 180 days. Comment on the issue otherwise it will be closed in 7 days