feat: node discovery via DNS

[Frando]

Description

This enables global node discovery over DNS, i.e. dialing nodes by just their node id.

Current setup is as follows:

• When dialing a node only by its NodeId, the new DnsDiscovery service is invoked. It will lookup a TXT record at (by default) _iroh_node.b32encodednodeid.testdns.iroh.link over regular DNS or DNS-over-http. Right now the Cloudflare DNS servers are configured. At testdns.iroh.link we run a custom DNS server
• Nodes publish their Derp address to this DNS server through Pkarr signed packets. This is an intermediate step, we decided that the publishing by default should not happen by the nodes directly but mediated through the Derp servers. Work for the latter happens in #2052

This PR thus allows for the following:

# terminal/computer 1
$ iroh console --start
Iroh is running
Node ID: qp2znfedwdij4llc5noizwfemfgba7bzxozvr4bp7hfsdmwqbpua
$ blob add ./myfile
...
Blob: o5uanh5s2zwn2sucy47puqidsfx2advxos7kajq3ajwitcwobhba
...

# terminal/computer 2
iroh console --start
blob get o5uanh5s2zwn2sucy47puqidsfx2advxos7kajq3ajwitcwobhba --node qp2znfedwdij4llc5noizwfemfgba7bzxozvr4bp7hfsdmwqbpua

Notes & open questions

• Misses node configuration in the CLI for the node origin domain (right now hardcoded to testdns.iroh.link). How do we want to expose this - CLI flag? Or in the config file? I'd say the latter.

• Offload publishing to the Derpers - see #2052

• Right now the records published via pkarr have a TTL of 30s - the iroh-dns-server will use that TTL as-is when serving the records over DNS. both can/should change?

• We can also very easily allow to lookup nodes not only by NodeId, but by any domain name. In the iroh-dns crate I included an example resolve that does just that. By setting a CNAME record you can even use any domain and simply point to the record hosted at the testdns.iroh.link server. So if, on your custom domain, you added a record like this

_iroh_node.frando.n0.computer CNAME _iroh_node.qp2znfedwdij4llc5noizwfemfgba7bzxozvr4bp7hfsdmwqbpua.iroh.link.

You can use this with the example to resolve to the node id and derp addresses:

cargo run --example resolve -- domain frando.n0.computer

Change checklist

• [x] Self-review.
• [x] Documentation updates if relevant.
• [x] Tests if relevant.

Closes #1248

[b5]

Does this close #1248?

[Frando]

Does this close #1248?

Yes

[Arqu]

/netsim

[github-actions[bot]]

feat/dns.d339b5c2a4e59023c5b00ba400a9afe445191ad4 Perf report:

test	case	throughput_gbps	throughput_transfer
iroh_latency_20ms	1_to_1	0.90	1.87
iroh_latency_20ms	1_to_3	3.11	6.46
iroh_latency_20ms	1_to_5	4.15	6.54
iroh_latency_20ms	1_to_10	5.28	6.43
iroh_latency_20ms	2_to_2	2.01	4.37
iroh_latency_20ms	2_to_4	4.08	7.49
iroh_latency_20ms	2_to_6	5.66	9.59
iroh_latency_20ms	2_to_10	7.53	11.11
iroh	1_to_1	1.12	2.21
iroh	1_to_3	2.82	5.93
iroh	1_to_5	4.28	6.86
iroh	1_to_10	5.32	6.57
iroh	2_to_2	1.84	3.69
iroh	2_to_4	4.10	8.00
iroh	2_to_6	5.62	9.37
iroh	2_to_10	7.71	11.44
iroh_latency_200ms	1_to_1	1.15	2.72
iroh_latency_200ms	1_to_3	2.88	5.92
iroh_latency_200ms	1_to_5	4.34	7.14
iroh_latency_200ms	1_to_10	5.43	6.76
iroh_latency_200ms	2_to_2	2.12	4.40
iroh_latency_200ms	2_to_4	4.09	7.94
iroh_latency_200ms	2_to_6	5.56	9.29
iroh_latency_200ms	2_to_10	7.59	11.14

[dignifiedquire]

@Frando I think this should also have an integration test, running the actual iroh-dns server

[Frando]

@Frando I think this should also have an integration test, running the actual iroh-dns server

~~Added an online test that uses dns.iroh.link for now. ~~
Edit: Removed the online test. The rate limits we set in the n0 pkarr relay are hit in CI. We should think about this some more actually: If we enable pkarr publishing by default, which this PR does, then our DNS server / pkarr relay will be hit a lot from CI (not only ours, but any CI that creates an Iroh node with the defaults). At least with the current, very low, limits. Of course we can increase the limits. But CI is very close to bad actor, when it comes to the "amount of publishes from a single IP address".

The PR also has an integration test with a very simple test server, see iroh-net/src/discovery.rs.

I will add a test that spawns an instance of the actual iroh-dns server code once we have a PR to move the server from https://github.com/n0-computer/iroh-dns-server/ into the iroh repo.