Add remote port forwarding subcommand

[atburke]

This change adds the remoteforward subcommand to Teleport that handles remote port forwarding, similar to how forwardv2 handles local port forwarding.

This is the flow for remoteforward:

• remoteforward is launched by the parent Teleport node with a Unix socket (labeled A) for inter-process commands (note: this part is not implemented in this PR).
• remoteforward reads from A an address and the file descriptor for a new socket B. remoteforward calls net.Listen with the given address and writes the listen address back to the parent over B (the address will have changed if the caller did not request a particular port, so the caller needs to know the port chosen).
• For each accepted connection on the listener, remoteforward creates a new socket C and writes C's file descriptor over B to the parent. remoteforward proxies between the accepted connection and C.

sequenceDiagram
    participant Parent
    participant Forwarder
    Parent->>Forwarder: A: { "127.0.0.1:0", [ fd(B) ] }
    Note right of Forwarder: l, err := net.Listen("tcp", "127.0.0.1:0")
    Forwarder-->>Parent: B: { "127.0.0.1:12345", [ ] }
    Note left of Parent: Parent replies to tcpip-forward<br>SSH request with chosen port<br>(not in this PR)
    Note right of Forwarder: conn, err := l.Accept()
    Forwarder-->>Parent: B: { "", [ fd(C) ] }
    Parent->Forwarder: C: (Proxy connection over C)

Part of #37117.

[espadolini]

Two concerns, one that can result in a simpler design and another that might end up trashing the whole thing:

• if we're shipping file descriptors back to the main Teleport process, why can't we just do that for the actual listening socket instead of doing it for each individual connection? it would let us actually manage a regular TCPListener/UnixListener that was still opened in the correct PAM context or whatever
• is there some way that we can guarantee that the unprivileged child process won't be able to mess with the file descriptors sent back to the parent? for example, I can think of some potential hangs in the runtime if a file descriptor thought to be nonblocking is set to blocking, and O_NONBLOCK is a property of the file description, meaning that the unprivileged child process is not prevented from keeping a file descriptor to the same socket to set it blocking afterwards

[atburke]

@espadolini I've updated the code to send the listening socket. As it's implemented now, it feels awkward to create a socket (B) on the parent side whose only purpose now is to respond with the listening socket and then close; do you think it would be worth it to just have one socket between the processes and multiplex the replies?

As for the file descriptor issue, would it make sense to use something like seccomp to prevent the child from changing the file mode after it's created?

[espadolini]

do you think it would be worth it to just have one socket between the processes and multiplex the replies?

Maybe we could add the functionality in the forward dialer? That one is currently one resident process per connection already (and could benefit from doing this socket transferring too, rather than forwarding data).

As for the file descriptor issue, would it make sense to use something like seccomp to prevent the child from changing the file mode after it's created?

I don't think that the go runtime would ever be happy with something like that.

It's possible that processes that begin their life as privileged end up not being ptraceable even after changing real uid/gid (unless the process itself decides to make itself so); if that's the case then we should be good.

[atburke]

Maybe we could add the functionality [of sending the dialing socket to the parent] in the forward dialer? That one is currently one resident process per connection already (and could benefit from doing this socket transferring too, rather than forwarding data).

@jentfoo Are there any consequences we should be aware of if we dial/listen as the user after doing PAM and then send the dialer's file descriptor to the parent, rather than proxying all of the data between the parent and the forwarding process?

[rosstimothy]

Friendly ping @jentfoo

[jentfoo]

This is a complex question, in short I do believe there is a risk here.

This was difficult to search for, so I am mostly looking at the man pages for fcntl. It sounds like some flags are in reference to the single process (for example FD_CLOEXEC), and others are referenced by the FD itself (for example O_APPEND or O_NONBLOCK).

So in theory it seems possible that if the FD is owned by the user they could reference it through proc and be able to manipulate these flags to cause impacts in other processes (even without the ability to manipulate the binary, which if that was possible a much wider range of attacks would be possible).

Can we duplicate the FD once passed to the privileged (parent) process, then validate the flags of the dup before use? That way flag changes to the original FD wont impact the copy our process has.

[espadolini]

@jentfoo: the transferred file descriptor is already a copy anyway, but the file description is the same, and that has some flags (such as O_NONBLOCK) that are shared through dup; I believe, however, that a process that starts privileged and then sheds privileges via setuid() is still not ptraceable by the new UID (unless it opts into it) - if that's the case, we can be reasonably confident about the behavior of the child process.

The question posed by @atburke is whether or not it's necessary to actually accept() connections in the child process, or if it's fine to go through the PAM session, create the socket, bind() it and listen() (which is what happens on a net.Listen), and then send the socket back to the parent for the accept() loop.

In both cases, and in the case where we can't even send the socket back but we need to copy data back and forth, all accounting would still be off, since it's either going to be sockets accept()ed but then sent to a different process with a different owner, or twice the file descriptors in use. From what I can tell, even in extremely fringe scenarios (like a PAM module putting the process in a different network namespace or something along those lines) it should be fine to accept() in a different process as long as bind() and listen() were called from the child in the PAM session, but I'm not an expert.

[jentfoo]

a process that starts privileged and then sheds privileges via setuid() is still not ptraceable by the new UID

I looked into this, and it appears generally you're correct. It appears that a process can use prctl(PR_SET_DUMPABLE, 1) after setuid in order to remain traceable. However this would require a binary replacement to be able to add the instruction. There are also some ways to override this behavior in SELinux and Yama LSM, however this would be non-standard so maybe excessive to consider.

[is it] necessary to actually accept() connections in the child process, or if it's fine to go through the PAM session, create the socket, bind() it and listen() (which is what happens on a net.Listen), and then send the socket back to the parent for the accept() loop.

I considered this more and another potential concern came to mind. It may also be possible for the user to interact with the communication channel passing the socket. Possibly allowing the user to replace the FD with another one that they control. I believe this may still be workable, to ensure the communication is as secure as possible. Specifically use SO_PEERCRED to authenticate the domain socket, and after receiving the FD validate all properties possible (for example validate it's a socket, and not a file or device).

Given the above, if we secure the channel, validate the socket, and make the assumptions about the system (for example security features are not disabled, and the binary is not replaceable), I believe passing the FD is workable and secure. But if there is not a compelling reason to pass the FD between the processes (like we are running short on open files, or performance issues), I probably would just use another socket since it's the simplest and most standard way to communicate across processes of different privileges.

@espadolini and @atburke let me know your thoughts and questions

[public-teleport-github-review-bot[bot]]

@atburke See the table below for backport results.

Branch	Result
branch/v15	Failed