💥 experimental: Iroh for webxdc statusUpdates

[Septias]

TODO:

• [x] Deduplicate webxdc updates (#5048)
• [x] Recreate gossips for reopening webxdcs
• [x] Store secret key in database
• [x] Destroy magicendpoint on stop_io() and create on start_io(), DC should have no connections after stop_io() finishes
• [x] ~Ensure that users stop receiving xdc updates when they are removed from the chat.~ (probably too difficult as the gossip topic aka Message-ID is the only secret and rotating it is hard, we should just state that anyone who received the message can still participate)
• [x] Add "ephermeral" flag into WebXDC updates, sending which results in not sending the message over SMTP. This will also make it easier to test that iroh works by having some tests which only send ephemeral messages.

[Septias]

@link2xt can you care about the cargo deny fail? I reverted the addition of n0-computers a second time because it felt redundant, but maybe that was actually needed?

[link2xt]

@Septias Ignore cargo-deny for now, it complains about duplicate dependencies and we may have these dependencies by the time this is merged. E.g. windows-* dependencies are always released as breaking changes and cause duplicate dependencies every time.

[link2xt]

Now it makes sense to rebase as probably all migrations for the next release are merged, dependencies mostly updated etc.

There is a security issue in the PR currently: we cannot use Message-ID as a gossip topic because Message-ID is not encrypted, which means the server which knows peer ID (from another chat, from mDNS or other source - it is not a private info) can join the gossip and read webxdc updates that are otherwise OpenPGP-encrypted.

[dignifiedquire]

we cannot use Message-ID as a gossip topic because Message-ID is not encrypted,

what about using the hash of the message-id?

[dignifiedquire]

@link2xt anything you are missing from the iroh side to move this forward?

[Septias]

I'm pushing this forward. We are currently focusing more on different milestones, so this experiment is halted for now. What do you need this for @dignifiedquire? I have merged my last PR, so I can also pick this up again.

[link2xt]

what about using the hash of the message-id?

Message-ID is public, so hash of Message-ID is public as well.

[dignifiedquire]

@Septias no specific need, just would love to see this land and want to make sure you are unblocked

[link2xt]

anything you are missing from the iroh side to move this forward?

No, I think DERP URLs was the only missing thing, then we can set up our own DERP server and point to it from provider DB or IMAP METADATA on chatmail. Otherwise what is missing is actually fixing this PR wrt. security and persistence of peer data without using a separate file.

[hpk42]

On Thu, Jan 18, 2024 at 07:20 -0800, Friedel Ziegelmayer wrote:

@Septias no specific need, just would love to see this land and want to make sure you are unblocked

if we include newer versions of Iroh crates with DC, we also will still have Iroh 0.4 for multi-device setup.
To minimize added app size it would help have a Iroh 0.4 release that updates as many dependencies as feasible to minimize redundant deps.

The general goal is how to get stability on the wire across DC/Iroh versions. For the multi-device transition to new Iroh we will likely ship two Iroh versions so that one can setup a new version from and old version of DC, and/or get good guidance in case of mismatches. When we drop Iroh 0.4 we'd hope that we can then use newer Iroh versions for a while without having to consider shipping 2+ versions in DC :)

[dignifiedquire]

I can take a look at reviving the update PR for device transfer, we can disable the relay parts, and so it would work the same as before. that way we can avoid the double iroh dep ideally

[Septias]

@adzialocha I guess it's important that we keep our secret key so that we can rejoin gossips after dc-restart? Also, see this https://github.com/n0-computer/iroh/blob/main/iroh-gossip/examples/chat.rs. I can do the rebase to the current main later if that helps.

[adzialocha]

@adzialocha I guess it's important that we keep our secret key so that we can rejoin gossips after dc-restart? Also, see this https://github.com/n0-computer/iroh/blob/main/iroh-gossip/examples/chat.rs. I can do the rebase to the current main later if that helps.

Yeah, right, if we want to reconnect again we have to make sure that the peer ids do not change

[adzialocha]

Notes

This little write-up will maybe be part of a future PR description. Still heavily WIP and experimental, not a real "final" spec.

WebXDC Ephemeral Channels API

The WebXDC API for developers will have two new methods sendEphemeral(payload: Payload) and setEphemeralListener((payload: Payload) => void) which allow them to explicitly send and receive data over an "ephemeral" channel. Payloads are JSON serializable types (number, string etc.) or UInt8Array. Ephemeral channels provide a fast transport layer, ideally directly between the machines who currently collaborate on the same WebXDC app together. Using ephemeral channels should be fast and feel almost instantaneous.

The downside of ephemeral messages is that they are not automatically persisted in contrary to regular, mail-based WebXDC messages. WebXDC application developers need to explicitly handle persistence if desired (for example by sending regular, persisting WebXDC messages via SMTP), but ideally only use ephemeral channels for non-critical data, like cursor positions.

Iroh

DeltaChat wants to use Iroh as an ephemeral channel provider, which attempts establishing direct p2p connections between peers (QUIC), using PlumTree as a gossiping protocol on top for efficient peer sampling. Iroh falls back to relay nodes when a direct connection can not be established. Even a relayed connection is faster than sending single WebXDC messages via mail.

Implementation

Ephemeral channels should lazily be established, to avoid bootstrapping p2p connectivity when it is not required for every running WebXDC app. By asking application developers to explicitly subscribe to or send ephemeral messages we can trigger Iroh to do the work for us.

In any case we need a way to agree upon a gossip topic id among all (future) participants, this topic id should never be guessable or readable for anyone outside the chat.

1. We introduce a new gossip_topic_id message header with a random 32 byte topic id, securely generated on the users device. This message header is encrypted and used in the same message when publishing a new WebXDC application in a chat group.

2. Whenever sendEphemeralor setEphemeralListener was called by the WebXDC application we start a routine to establish p2p connectivity and join the gossip swarm with Iroh

3. This routine requires a peer to "announce" themselves, other peers need to learn about their peer id first, before they can connect. A regular WebXDC message is sent which contains this peers identifier. We need to make sure to not contain any other information, like IP addresses in this information (this is slightly different to the "ticket invite" flow Iroh suggests)

4. After announcement this peer joins the gossip swarm with an empty list of peer ids (as they don't know anyone yet)

5. Upon receiving this WebXDC "announce" message of a new peer, the other peers store this new peer id in the database (scoped per WebXDC app instance / message id) for the future

6. As soon as the other peers learned about this new peer id they add it to their Iroh endpoint and gossip swarm, and join it as well if they haven't yet

7. After connection with the gossip swarm was successfully established, the actual payload of sendEphemeral is sent or we're subscribed to the incoming stream with setEphemeralListener

8. When entering the app again and using one of the ephemeral channel API methods again, the same procedure repeats, but the peer can also add previously learned peer ids to their known peer list

Caveats / Open Questions

• When a user got removed from a chat, they will still be able to receive messages on the ephemeral channel as they've learned about the topic id
• When the app is closed we need a way to leave the swarm, currently there's no way to determine if a WebXDC app was closed

[Septias]

superseded by #5346