feat(iroh-net): Removeable discovery

[rklaehn]

Description

Adds a version of ConcurrentDiscovery where the individual discovery services are identified by ids.

That way you can dynamically add them, remove them, and even get a reference.

Breaking Changes

None, this just adds something. In the long term we would probably deprecate and then remove ConcurrentDiscovery however, since the functionality is quite overlapping.

Notes & open questions

The current implementation uses an Arc<Mutex<HashMap>>. When using the services in the service trait implementation, we hold the lock while e.g. calling publish. This is OK for a properly implemented discovery service that does not do anything expensive during publish, but you could come up with a convoluted discovery impl that would e.g. try to remove a discovery service during publish, which would cause a deadlock. Also, people might block in publish, meaning that another code path that calls remove would be blocked.

Sooo. The safe solution for this is to not hold the lock at all while executing code that can be influenced by the user. To do that there are two approaches:

• make a deep copy. Not the end of the world, but also not nice. You would allocate during every call to resolve.
• use a persistent data structure for the inside of the Arc<Mutex<...>> and do a zero cost snapshot. This means having a dependency on rpds.

What do you think?

1. hope that people won't do totally broken and convoluted impls of discovery
2. do a defensive deep copy and eat the overhead
3. use persistent data structures

Change checklist

• [ ] Self-review.
• [ ] Documentation updates following the style guide, if relevant.
• [ ] Tests if relevant.
• [ ] All breaking changes documented.

[github-actions[bot]]

Documentation for this PR has been generated and is available at: https://n0-computer.github.io/iroh/pr/2827/docs/iroh/

Last updated: 2024-11-20T10:48:44Z

[github-actions[bot]]

Netsim report & logs for this PR have been generated and is available at: LOGS This report will remain available for 3 days.

Last updated for commit: fc640ca

[dignifiedquire]

if you just store an Arc<DiscoveryServiceEntry> you can clone super cheap, without a lot of overhead

[rklaehn]

if you just store an Arc<DiscoveryServiceEntry> you can clone super cheap, without a lot of overhead

Yeah, but then I still have to make a copy of the Vec that contains them. I already had a version where I was using a smallvec for this, but using a persistent data structure seems like a better solution.