canonical
`remoteproxy` for remote containers
One of major problems with remote LXD/LXC containers that prevents their adoption is inability to map service ports running in remote container to local machine with LXC client (mapping other way is also needed).
The common use case for secure port forwarding is sharing local project directory with remote container. Doing that requires establishing additional secure channel and routing rule for NFS, SSH or 9p/9p2000 protocol as in the issue above. Using 9p also requires putting encryption layer, and there are no easy solutions like Let's Encrypt for domain names.
LXD already provides a secure channel, so this issue it to reuse that for port forwarding. I asked the question on the forum 20 days ago and there are still no replies.
The proposal had been raised a year ago in #5414, where is was marked as isn't something that LXD is meant to do, at least for now. which was strange, because there was already proxy device for forwarding tcp connections. I cam accept if LXD was not meant to support port forwarding, or not meant to support remote containers, but since it is now supports both, it is only logical to add the missing link.
Issues that made LXD is now ready to share its connection for port forwarding on remote containers:
- presence of
forkproxyexecuting host side processes for container interaction, which can be used to maintain connection with remote LXD client and do simple channel copy - backgroud process manager for managing
forkproxy,forkdnsanddnsmasq
I could probably do the patch myself, but I need pointer how to a read and write to connection between LXD client and server.
Marking as a Maybe as this would be a LOT of work to get this done when you can instead just use ssh, possibly with a jump through the host (-J option) combined with its port forwarding to achieve effectively this.
As far as LXD is concerned, this would need:
- CLI command to redirect some ports/sockets from/to the instance, somewhat similar feature set to the proxy device.
- Send the bi-directional, multi-connection traffic over some kind of websocket connection.
- On the server side, spawn and attach a process into the container's network namespace to list or connect as needed.
The socket over websocket logic is the obvious complex issue here as we need to establish one connection per redirected port or socket but then allow multiple connection over that so that things work as expected. SSH does this by having a complex channel mechanism as part of the remote shell connection which allows for effectively detecting new connections on either side and establishing new channels as needed. This may be a suitable approach for us too, though as we have the extra step of entering the container (or virtual machine through the agent), this would in a trivial implementation mean one sub-process per connection.
That in turn may make things very weird and brittle as a re-implementation of what ssh does with the way containers work would effectively mean that accessing a web page over such a redirected port, in the event where the web page includes a number of other resources coming from the same redirected port (images, videos, css, ...) would mean up to 10 connections (normal browser limit), in turn meaning up to 10 processes running on the LXD host, spawning/dying constantly as each request is established and disconnected.
This could cause a LOT of forked sub-processes doing expensive container operations (namespace switches), going all the way to taking down LXD or significantly slowing down the target host and kernel by running something as simple as a recursive wget over such a redirected port.
So anyway, it seems likely we'd need a pretty custom solution so we can cheaply establish/close additional connections on a particular port, making any of this practical. Doing this in a way which works on all platforms (the client runs on Windows, Mac and Linux) and works in both containers and VMs on the LXD side will be quite a bit of work and not something we're likely to be putting on our roadmap at this stage.
@tomponline I think we can keep it around for actual port redirection over the LXD API. So letting you pass tcp/80 from 127.0.0.1 in the container to some port on your machine or the other way around.
Ah yeah that would be nice. Perhaps by first moving as much of forkproxy to Go as we can (like forkfile) might also fix those intermittent but frequent test failures.