Creation of Tor hidden services

[bonomat]

The PR in #438 sparked some discussions about how Tor can be integrated differently. In particular, the discussion is about how and where hidden services should be created.

The current implementation interacts with Tor from an application level point of view where the underlying network does not need to know about Tor nor hidden services if the purpose is only to listen.

For that, hidden services are registered from the ASB's application point of view and not from the network point of view. Concretely, the hidden services are currently created during start-up from within asb.rs: https://github.com/comit-network/xmr-btc-swap/blob/00b40c4a8fd92276d89517fcac4ee311f68e4b3e/swap/src/bin/asb.rs#L110-L113

The question is whether Tor should be treated as a transport or not and if the hidden services should be created from within the network part of the code-base.

Note: If an application also needs to dial, the situation is differently, i.e. the network needs to know about how to dial addresses through the Tor socks5 proxy.

This ticket is explicitly about the listening and the creation of hidden services.

---

To summarize some points which were raised:

• A Transport's responsibility is to dial and listen using a certain transport protocol. If you have a Tor-Transport, it seems perfectly reasonable that the implementation somehow achieves that it is reachable via Tor. One way of doing this is setting up an ephemeral service. There are other ways as well so we will probably need some configuration at some point.

Concretely proposed was to create hidden services from within Swarm.listen(...) if an onion wildcard address was provided in the config. E.g. /onion3/CANIHAZNEWEPHEMERALHIDDENSERVICEPLZZZZZZZZZZZZZZZZZZZZZZ:65535

Originally posted by @thomaseizinger in https://github.com/comit-network/xmr-btc-swap/pull/438#discussion_r618823503

• If we ever add support for predefined hidden services, simply replacing that wildcard address with the address you want to use is much more intuitive then adding a dedicated configuration section.

Refers to the idea to allow users to configure from outside of the application and don't let the application register hidden services. As of now a Tor hidden service will always be created if Tor's control port is reachable. The control port is an extra field in the config which defaults to Tor's default control port.

Originally posted by @thomaseizinger in https://github.com/comit-network/xmr-btc-swap/issues/438#issuecomment-828876512

[bonomat]

Some thoughts about this topic:

I think we should treat Tor integration w.r.t the creation of hidden services from two different perspectives:

1. Configuration: The user point of view: what is it the user has to configure to have or to not have hidden services created
2. Code: The developer point of view: how should this be implemented in the code base.

Configuration/Setup

Personally, I would like to have a hidden service created if Tor can be found. If not, I'm either fine creating hidden services myself or I don't want hidden services at all. Since I see privacy as important, from the configuration I'd like to default to the creation of hidden services.

I'm not sure if it was explicitly proposed, but I would find it weird if I had to put a wildcard onion address into the config. From how I used Tor so far, this does not seem to best practice neither.

Bitcoin for example let's you configure a few things:

# Bind to given address and always listen on it. (default: 0.0.0.0). Use [host]:port notation for IPv6. Append =onion to tag any incoming connections to that address and port as incoming Tor connections
bind=0.0.0.0
# Automatically create Tor hidden service.
listenonion=1
# Only connect to peers via Tor.
onlynet=onion
# Tor control <ip:port> to use if onion listening enabled.
torcontrol=127.0.0.1:9051
# Use separate SOCKS5 proxy <ip:port> to reach peers via Tor hidden services.
onion=127.0.0.1:9050

Bitcoind will create a tor hidden service by default if Tor is reachable under the torcontrol address. It will by default listen on 0.0.0.0. Additionally you can configure bitcoind to only send message out through the Tor socks5 proxy (onion).

These default seem to work for people who want to run bitcoind behind Tor.

As of now, our configuration is similar (only focusing on the listening part as the ASB does not make outgoing connections).

[network]
listen = ["/ip4/0.0.0.0/tcp/9939", "/ip4/0.0.0.0/tcp/9940/ws"]
[tor]
control_port = 9051
socks5_port = 9050

Same as for Bitcoind, the only way to disable the creation of hidden services is to set an invalid control port or stop Tor on your machine.

Code

//reserved// might be added later

[thomaseizinger]

I'm not sure if it was explicitly proposed, but I would find it weird if I had to put a wildcard onion address into the config. From how I used Tor so far, this does not seem to best practice neither.

Can you elaborate in more detail please on what exactly is "weird" about it? Contrary to weird, I find it very elegant and see two big benefits:

• Consistency: The user uses the same interface (the listen array) to control how their application is reachable by others, regardless of the transport technology (TCP, WS, Tor)
• Versatility: No additional configuration options are required (apart from Tor's control port) and yet, we can support a variety of use cases:
  • Disabling Tor can be achieved by removing the listen address
  • Hooking into an existing hidden service can be achieved by using the existing hidden service address
  • Creating an ephemeral hidden service can be achieved through the use of a wildcard address (concrete format of wildcard address subject to bike-shedding) or some other kind of "marker" although a wildcard address borrows conceptually from the idea of listening at 0.0.0.0 so it is not particularly new.

I am more than happy to roll with a different solution as long as it provides similar or even better properties. I just don't see how the current solution is better when measured on scales like consistency and size of configuration surface.

---

Not so important answers to some of your points, included for completeness (I would like to focus on the above as I think it is more relevant):

I'm not sure if it was explicitly proposed, but I would find it weird if I had to put a wildcard onion address into the config. From how I used Tor so far, this does not seem to best practice neither.

I am curious. What other ways did you find where people integrate libp2p and Tor when you talk about these best practices? Just looking at other applications that support Tor is not a fair comparison really because it misses the main point of this discussion: the exposed configuration surface.

Since I see privacy as important, from the configuration I'd like to default to the creation of hidden services.

Defaulting this is easy, we can make the wildcard address part of the default value of listen if it is not specified in the config file.

These default seem to work for people who want to run bitcoind behind Tor.

I would seriously question whether any configuration surface and API of bitcoind is by any means ideal and should serve as a role model :D In my experience, anything UX in regards to bitcoind is often constrained by some things being hard in C++ which is a constraint that doesn't apply to us. Also, without a demonstrated feedback loop and evaluation process, you can't actually say that it "works for people" because they might also just accept what they are given, even if it is far from ideal.

Same as for Bitcoind, the only way to disable the creation of hidden services is to set an invalid control port or stop Tor on your machine.

I would argue that this is more of a bug than a feature as it stems from bad modelling of the configuration options.

[thomaseizinger]

After several more discussions and experiments, I believe a better way of integrating with Tor is to:

• Require users to set up a Tor hidden service outside of the application
• Optionally provide a sub-command on the application that can help with setting up such a hidden service by - for example - writing to torrc directly or printing what the user should add to it
• Minimize the configuration options for the user in which network interfaces are being used to a simple choice between "clearnet" and "tor". "Tor" would be the default and fail on startup if the Tor daemon cannot be reached. Additionally, we would try and read the hidden service configuration by default and set up a matching local listening socket: i.e. if the configured hidden service is on local host 1337, we would pick that up and automatically listen on 127.0.0.1:1337 to get all the traffic. It should also be possible to override this "auto-detection" with manual parameters in case the user doesn't want to give read permissions to the config file.

Overall, the theme of this approach is:

• Configuration simplicity with a focus on user's usecase
• No magic where starting the daemon "does all the things"
• Convenient integration. If things are left at their defaults, using things should be easy.
• Composability: Provide tools / commands that help with getting things configured where the user can always choose to do something "manual" if they want to