Design for "remote" qube type

[marmarek]

How to file a helpful issue

The problem you're addressing (if any)

As an initial step for Qubes Air, there needs to be a way to execute qrexec call into a qube that is running on another system.

The solution you'd like

Create a new "remote" qube type. Such qube can be a target for a qrexec calls, but otherwise have little real properties. It should be flexible enough so the actual remote target can be on another qubes sytem (a qube there) but can be also a non-qubes VM (as long as has appropriate software installed) or not a VM at all (for example, a raspberry pi).

The qrexec connection to remote qube should be performed by a separate "proxy" qube, using a qrexec service for that. One proxy may be used for multiple remote qubes, but there can be also multiple proxy qubes on one system. The proxy qube is responsible to enforce qrexec connection properties, especially confidentiality and integrity (by using some encrypted channel). Choosing specific crypto solution is out of scope for this ticket (and in fact, different solutions may be used in different cases).

Similarly, remote qube should be able to also initiate qrexec calls. Those will be received by the proxy qube, and then forwarded to requested target (which then goes through usual policy evaluation). The qrexec protocol should be extended to allow the proxy qube executing calls on behalf of remote qubes. The proxy qube should be allowed to identify as qubes for which it works as a proxy. It's yet to be determined what should happen if it tries to pretend to be some different source, possible options include: refuse the call (IMO preferred), treat the call as from the proxy itself, introduce another source type like "unknown source" (that includes information from which proxy it was received).

And finally, there should be some registration mechanism that allows the user to conveniently add new remote qubes. Besides setting basic remote qube metadata (to be established what exactly) and which proxy to use, it should also allow configuring transport mechanism for the proxy in the same step (likely by having a field for some proxy-specific opaque data). One idea for interactive remote qube registration is something like magic-wormhole (so, ask the user to type some passphrase on the other device to establish a secure channel for the actual registration protocol).

The point of this issue is to design all relevant interfaces, including:

• what properties a remote qube should have
• how proxy qrexec service should look like (arguments, payload, how the information about the actual proxied call is transferred)
• how proxy qube should be configured, where should the connection-related settings be stored (things to consider: should disposable proxy qube be supported? but also, avoid sensitive data in qvm-features etc)
• what information proxy qube should receive, how exactly the remote qube setup should work (what services are involved, what formats they use)
• how receiving qrexec call should look like, how the information on the original (remote) qube should be transferred
• are there any qrexec policy changes necessary?

The value to a user, and who that user might be

At the initial stage, this will allow making qrexec calls between specific qubes on different physical machines. This should allow for example copying files via qvm-copy to another system.

[fepitre]

Adding here what we did with @HW42: https://gist.github.com/fepitre/7af571bbc9de86824e543e2fd49be0aa

[fepitre]

Based on some iterations in direct chat with @marmarek, here's an outline of the design for Qubes Air:

• Overall Architecture:

  • Communication flow: (local_proxy, local_qube) -> (remote_proxy, remote_qube)
  • local_proxy/remote_proxy can be identified as a host: qube, VM, or system.
  • remote_qube could be a qube or "local" (e.g. local system of the remote_proxy like RPI, KVM VM, etc.)

• Components:

  • Local Proxy: Handles communication with the remote proxy.
  • Remote Proxy: Handles communication with the target (remote_qube).

• Transport Mechanisms:

  • SSH/API/SOCKET for communication between proxies.
  • Different RPC methods in the local proxy to initiate calls (e.g., SSHProxy, WireguardProxy).

• Proxying Services:

  • Consider defining a standard naming convention for proxying services (e.g., qubesair.SSHProxy, qubesair.WireguardProxy).

• Qubes Types:

  • Introduce new qubes types: LocalVM and RemoteVM.
  • BaseVM as the foundation, but with modifications to accommodate the new types.

• State Management:

  • Remote qube state (powered on/off) may be out of scope for standard RemoteVM but could be considered for specific implementations.

• Code Refactoring:

  • Evaluate existing code (e.g., BaseVM and QubesVM) to determine what functionalities are relevant for RemoteVM.

• RPC Policy Resolution:

  • Resolve RPC policies with translation for final calls to be made.

• Proxy Relations:

  • Proxy relationships should be real properties of RemoteVMs.
  • Avoid overcomplicating by chaining proxies; if necessary, proxy to another RemoteVM.

• Proxy Settings and Secrets Management:

  • Consider storing sensitive data (e.g., SSH private key, Wireguard key, API token) securely, possibly in the /rw directory.
  • Manage proxy settings (including configurations, name, keys) within /rw of proxy qube.

• Mapping and Settings:

  • Maintain mapping between proxies and connection settings within RemoteVM entities.

[fepitre]

Preliminary notes:

What properties a RemoteVM should have:

• uuid
• qid
• label
• name (possibly with an alias being "local_name" if needed)
• local_proxy
• remote_proxy (information to be passed to local_proxy qube)
• remote_name
• proxy_method (e.g. SSHProxy, WireguardProxy)

[fepitre]

How proxy qrexec service should look like (arguments, payload, how the information about the actual call is transferred):

[3hhh]

qubes-qrexec-proxy has a pretty similar architecture and it should be fairly straightforward to implement a source & destination plugin for ssh with it (destination = ssh server, source = ssh client). Back then I just didn't implement anything network related as I was more interested in local mitigations against side channel attacks over qrexec. Admittedly I haven't tested it anymore since 4.0 or so, but I guess it should still work.

Even if you don't like my approach, it would be great if you could make your code extensible to support advanced functionality such as qrexec sniffing, protocol modifications such as one-way traffic, streamlining, ...

[marmarek]

Even if you don't like my approach, it would be great if you could make your code extensible to support advanced functionality such as qrexec sniffing, protocol modifications such as one-way traffic, streamlining, ...

The design has specifically configurable transport service, so 3rd party transport implementations are possible (including stacking extra stuff on top of 1st party one). You may want to watch qubes air talk on qubes summit on Friday :)