WebRTC Exploration

[D4nte]

Problem

Some application may need a low latency (<100ms) connectivity, see https://github.com/vacp2p/rfc/issues/446. .e.g games, voice call, video calls.

Solution

WebRTC is a common way to achieve this low latency in a Browser environment. Investigate and add WebRTC support.

Definition of Done

• [x] WebRTC Direct PoC PR as part of js-waku, probably with NodeJS helper
• [x] Requirements for nim-waku are outlined
• [x] Potential issues to fix upstream are opened
• [ ] Browser-to-Browser WebRTC PoC. Blocked on #681

Notes

This may be relevant: Check out Status how they do things on secure transport file (compact code) and bundle retrieval https://specs.status.im/spec/5#bundle-retrieval

[oskarth]

Probably not very high priority yes. In terms of order of operations, before this there is:

1. Relay + query Store in Node
2. Relay in Browser (via Node)
3. Websocket support for Browser => query Store in Browser

And then WebRTC for Browser to Browser (probably informed by specific user story that Dapp etc have)

[D4nte]

Probably not very high priority yes. In terms of order of operations, before this there is:

1. Relay + query Store in Node

2. Relay in Browser (via Node)

3. Websocket support for Browser => query Store in Browser

And then WebRTC for Browser to Browser (probably informed by specific user story that Dapp etc have)

Yes, this is in line with the position it has in the project. Thanks!

[D4nte]

Let's start with https://github.com/libp2p/js-libp2p-webrtc-direct as it may help getting rid of ssl certificates on nim-waku side.

[oskarth]

Relevant https://blog.ipfs.io/2021-06-10-guide-to-ipfs-connectivity-in-browsers/

[D4nte]

WebRTC Direct works in Chrome but not Firefox. There are some issue with Chrome to investigate. The Firefox issue is reproduced with https://github.com/libp2p/js-libp2p/tree/master/examples/webrtc-direct

14:17:11.471 libp2p:webrtcdirect connection opened 127.0.0.1:9090 +5s common.js:113:9
14:17:11.472 libp2p:ip-port-to-multiaddr:err invalid ip:port for creating a multiaddr: c67a8623-319c-4b31-96fa-e35c6b25a83a.local:58088 +10s common.js:113:9
14:17:11.472 libp2p:dialer token 1 released +5s common.js:113:9
14:17:11.473 libp2p:dialer:err AggregateError@http://localhost:1234/index.93e19648.js:42430:9
maybeSettle@http://localhost:1234/index.93e19648.js:42373:24
.hLj2A</pSome/</<@http://localhost:1234/index.93e19648.js:42394:39
async*.hLj2A</pSome/<@http://localhost:1234/index.93e19648.js:42396:15
.MqLlq</PCancelable/this._promise<@http://localhost:1234/index.93e19648.js:42534:20
PCancelable@http://localhost:1234/index.93e19648.js:42509:25
pSome@http://localhost:1234/index.93e19648.js:42355:36
["6p5iW"]</module.exports@http://localhost:1234/index.93e19648.js:42335:32
run@http://localhost:1234/index.93e19648.js:42257:26
_createPendingDial@http://localhost:1234/index.93e19648.js:42096:34
connectToPeer@http://localhost:1234/index.93e19648.js:41975:75
async*_autoDial@http://localhost:1234/index.93e19648.js:37850:47
Async*.gwg31</Retimer/this._timerWrapper@http://localhost:1234/index.93e19648.js:33295:26
setTimeout handler*Retimer@http://localhost:1234/index.93e19648.js:33298:23
retimer@http://localhost:1234/index.93e19648.js:33330:12
_autoDial@http://localhost:1234/index.93e19648.js:37858:33
Async*start@http://localhost:1234/index.93e19648.js:37810:14
_onDidStart@http://localhost:1234/index.93e19648.js:8822:33
async*start@http://localhost:1234/index.93e19648.js:8592:24
async*.aLC0o</<@http://localhost:1234/index.93e19648.js:576:18
async*.aLC0o<@http://localhost:1234/index.93e19648.js:528:10
newRequire@http://localhost:1234/index.93e19648.js:71:24
@http://localhost:1234/index.93e19648.js:122:15
@http://localhost:1234/index.93e19648.js:145:3
 +10s common.js:113:9

[D4nte]

WebRTC Direct works in Chrome but not Firefox. There are some issue with Chrome to investigate. The Firefox issue is reproduced with libp2p/js-libp2p@master/examples/webrtc-direct

14:17:11.471 libp2p:webrtcdirect connection opened 127.0.0.1:9090 +5s common.js:113:9
14:17:11.472 libp2p:ip-port-to-multiaddr:err invalid ip:port for creating a multiaddr: c67a8623-319c-4b31-96fa-e35c6b25a83a.local:58088 +10s common.js:113:9
14:17:11.472 libp2p:dialer token 1 released +5s common.js:113:9
14:17:11.473 libp2p:dialer:err AggregateError@http://localhost:1234/index.93e19648.js:42430:9
maybeSettle@http://localhost:1234/index.93e19648.js:42373:24
.hLj2A</pSome/</<@http://localhost:1234/index.93e19648.js:42394:39
async*.hLj2A</pSome/<@http://localhost:1234/index.93e19648.js:42396:15
.MqLlq</PCancelable/this._promise<@http://localhost:1234/index.93e19648.js:42534:20
PCancelable@http://localhost:1234/index.93e19648.js:42509:25
pSome@http://localhost:1234/index.93e19648.js:42355:36
["6p5iW"]</module.exports@http://localhost:1234/index.93e19648.js:42335:32
run@http://localhost:1234/index.93e19648.js:42257:26
_createPendingDial@http://localhost:1234/index.93e19648.js:42096:34
connectToPeer@http://localhost:1234/index.93e19648.js:41975:75
async*_autoDial@http://localhost:1234/index.93e19648.js:37850:47
Async*.gwg31</Retimer/this._timerWrapper@http://localhost:1234/index.93e19648.js:33295:26
setTimeout handler*Retimer@http://localhost:1234/index.93e19648.js:33298:23
retimer@http://localhost:1234/index.93e19648.js:33330:12
_autoDial@http://localhost:1234/index.93e19648.js:37858:33
Async*start@http://localhost:1234/index.93e19648.js:37810:14
_onDidStart@http://localhost:1234/index.93e19648.js:8822:33
async*start@http://localhost:1234/index.93e19648.js:8592:24
async*.aLC0o</<@http://localhost:1234/index.93e19648.js:576:18
async*.aLC0o<@http://localhost:1234/index.93e19648.js:528:10
newRequire@http://localhost:1234/index.93e19648.js:71:24
@http://localhost:1234/index.93e19648.js:122:15
@http://localhost:1234/index.93e19648.js:145:3
 +10s common.js:113:9

Opened a PR: https://github.com/libp2p/js-libp2p-webrtc-direct/pull/147

[D4nte]

Interesting reads:

• https://github.com/libp2p/js-libp2p/issues/385#issuecomment-514300699
• https://github.com/libp2p/js-libp2p-webrtc-direct/issues/98

[D4nte]

WebRTC Direct findings:

• It is possible to connect to a remote node using WebRTC Direct, ~without the need of setting up any certificates~. This works with both dns4 and ip4 addresses. Tested with https://github.com/libp2p/js-libp2p/tree/master/examples/webrtc-direct?rgh-link-date=2022-02-28T04%3A21%3A19Z
• It does not work in Firefox due to https://github.com/libp2p/js-libp2p-webrtc-direct/pull/147

Next: integrate and test in js-waku

[D4nte]

WebRTC Direct is promising as an alternative, out-of-the-box transport protocol for js-waku<>nim-waku:

• https://github.com/status-im/js-waku/pull/588
• https://github.com/status-im/nim-waku/issues/872

[oskarth]

Linking related upstream issues:

• https://github.com/status-im/nim-libp2p/issues/408
• https://github.com/libp2p/specs/issues/220

[oskarth]

@D4nte just so you are aware that WebRTC Direct isn't being actively pursued on a libp2p protocol level:

(aka libp2p-webrtc-direct)

Note that webrtc direct will likely not be promoted to the general libp2p WebRTC protocol, given that it requires a previous WebSocket connection to exchange the SDP.

The goal of the upcoming design (https://github.com/libp2p/specs/issues/220) is to not require a previous exchange.

Happy to expand on this in case you are planning on working on this any time soon.

From https://github.com/status-im/nim-libp2p/issues/698#issuecomment-1059126459

[D4nte]

@D4nte just so you are aware that WebRTC Direct isn't being actively pursued on a libp2p protocol level:

Indeed. Thanks.

WebRTC direct is an interesting protocol because it enables to connect to a server without needing for this server to have a SSL certificate configured. ~It seems that this exact property is what makes WebRTC not a good candidate for libp2p specs as they want the protocol to provide a security layer without adding another (I guess noise). I find it surprising because I always expect to run noise within a transport protocol but maybe that's not the expectation. ~

Yet, to negotiate the handhake, information between the browser and server must be exchanged. In libp2p's webrtc-direct, this is not via wss, meaning a certificate setup is needed.

What is interesting and not the first time I have heard of it, is the intent for protocol labs/libp2p bootstrap nodes to generates certificates for nodes to fulfill this requirement. In this case, WebRTC Direct is not attractive anymore and something much more efficient such as WebTransport can be used (a kind of evolution of WebSocket).

[D4nte]

Browser-to-Browser WebRTC Exploration findings

Note: This is about WebRTC, browser-to-browser communication. It is different from WebRTC Direct, which is browser-to-server communication. Most comments above are about the exploration of WebRTC Direction

Goal

libp2p's Browser-to-browser WebRTC needs a signaling server to which both browsers can connect. Both browsers must connect to the same signalling server.

This is needed to enable the browsers to exchange handshake data to setup the WebRTC connection. The browsers use each other peer id to identify their counterpart and tell the signal server to whom a message should be forwarded to.

The goal of this exploration was to determine whether it would be possible to remove the necessity of a signalling server and instead, use the Waku network to enable two browser to negotiate a WebRTC connection.

Result (tl;dr:)

• js-libp2p WebRTC library are functioning (as far as I could test)
• It seems plausible to indeed make do of the signalling server
• But I was not able to produce a PoC within a 2-3 days timebox
• Next actions to reach the initial goal have been outlined in this issue description: Build a Waku Socket PoC.

Details

The PoC

The PoC was not successful because I tried to take a number of hacky design decision on the fly to just make it work, especially around using Waku as a socket server. Which, at the end of the day, is mostly what a signalling server is.

I believe we can be successful in replacing the signalling server with the Waku network if we create few building bricks first. See the last section of this comment.

Proof of Code: https://github.com/D4nte/js-libp2p-webrtc-star/tree/transport-waku

The Signalling Server

The WebRTC Star Signalling Server is a socket server (using socket.io) that handles the following events:

interface SocketEvents {
  'ss-handshake': (offer: HandshakeSignal) => void
  'ss-join': (maStr: string) => void
  'ss-leave': (maStr: string) => void
  'ws-peer': (maStr: string) => void
  'ws-handshake': (offer: HandshakeSignal) => void
  'error': (err: Error) => void
  'listening': () => void
  'close': () => void
}

Connection Management

• ss-join: Peer tells the server they are joining
• ss-leave: Peer tells the server they are leaving
• disconnect: Peer has disconnected (emitted by socket server)

Once a peer has joined, the signalling server regularly send the current list of connected peers to it by emitting ws-peer events.

WebRTC Handshake

A peer connected to the signalling server can initiate a handshake with another peer by emitting a ss-handhake event with the following payload:

export interface HandshakeSignal {
  srcMultiaddr: string
  dstMultiaddr: string
  intentId: string
  signal: Signal
  answer?: boolean
  err?: string
}

The signalling server then does some sanity checks and forward the offer to the right peer (dstMultiaddr) by emitting a ws-handshake event to said peer. Then the other peer can reply to the offer and in turn, emit a ss-handshake event to the signal server, which forwards it with a ws-handhake event.

sequenceDiagram
	actor Alice
	participant "Signal Server"
	actor Bob
	Alice->>"Signal Server": `ss-handshake` offer
	"Signal Server"-->>Bob: `ws-handshake` offer
	Bob->>"Signal Server": `ss-handshake` answer
	"Signal Server"-->>Alice: `ws-handshake` answer

Waku Socket Server

My first approach to replace the signalling server with Waku was to create a Waku Socket Server that provides the same API than a socket server but over Waku: https://github.com/D4nte/js-libp2p-webrtc-star/blob/transport-waku/packages/webrtc-star-transport/src/waku-socket.ts

I believe this is where I failed to properly design this on the fly has some considerations around content topics and peer discovery must be made.

The signaling server provides the ability for a given peer to communicate with an other peer by peer id. I try to skip this step by doing only working with a 2 peers scenario but it did not work.

Action item

To enable WebRTC browser-to-browser connection, we must first design and build a protocol to enable two peers to communicate over Waku given their peer id. Considering the recent work done with Noise: https://rfc.vac.dev/spec/35/ I think it would be valuable to create such a protocol/software to establish a secure channel (ie, not direct connection) between two parties.

Once this is done then it should be trivial to use this instead of a signal server for WebRTC handshake.

Tracked with https://github.com/status-im/js-waku/issues/681

WebRTC Star libp2p transport

To integrate WebRTC transport in Waku, it needs to be used with libp2p. For this, it needs to implement the libp2p transport interface:

export interface Transport {
  /**
   * Used to identify the transport
   */
  [Symbol.toStringTag]: string

  /**
   * Used by the isTransport function
   */
  [symbol]: true

  /**
   * Dial a given multiaddr.
   */
  dial: (ma: Multiaddr, options: DialOptions) => Promise<Connection>

  /**
   * Create transport listeners.
   */
  createListener: (options: CreateListenerOptions) => Listener

  /**
   * Takes a list of `Multiaddr`s and returns only valid addresses for the transport
   */
  filter: MultiaddrFilter
}

The methods of interest are:

• dial
• createListener

Let's review how they are implemented for libp2p-webrtc-star-transport.

createListener

createListener MUST return a Listener:

export interface Listener extends EventEmitter<ListenerEvents> {
  /**
   * Start a listener
   */
  listen: (multiaddr: Multiaddr) => Promise<void>
  /**
   * Get listen addresses
   */
  getAddrs: () => Multiaddr[]
  /**
   * Close listener
   *
   * @returns {Promise<void>}
   */
  close: () => Promise<void>
}

The interesting part is the listen function. Here is an example of the kind of multiaddr that listen (for webrtc-star) accepts:

/dns4/wrtc-star1.par.dwebops.pub/tcp/443/wss/p2p-webrtc-star

When called, listen connects to the Signal Server specified in the passed multiaddr (e.g. /dns4/wrtc-star1.par.dwebops.pub/tcp/443/wss). It then setups a number of hooks to translate Signal Server events to the equivalent in libp2p transport terms. Once connected to the signal server, it emits a ss-join to register its id with the server.

dial

Here is an example of the kind of multiaddr that dial accepts:

/ip4/188.166.203.82/tcp/20000/wss/p2p-webrtc-star/p2p/QmcgpsyWgH8Y8ajJz1Cu72KnS5uo2Aa2LpzU7kinSooo2a

• The first part /ip4/188.166.203.82/tcp/20000/wss provide connection details to the signal server.
• p2p-webrtc-star specifies the transport protocol
• p2p/QmcgpsyWgH8Y8ajJz1Cu72KnS5uo2Aa2LpzU7kinSooo2a identifies the peer we are connecting to (the other browser).

When dial is called, it:

• Checks that we are already connected to the signal server specified in the multiaddr: Meaning the same signal server must be passed to the listen function beforehand.
• Execute the [WebRTC Handshake](WebRTC Handshake).

https://github.com/status-im/js-waku/issues/681 Also tracks making a libp2p transport that uses Waku Socket.

WebRTC over Waku

We have reviewed that a signalling server logic is pretty straightforward: It enables two connected peer to communicate with each other; it has some forwarded logic that is specialized for WebRTC handshakes; peer ids are used as a mean of peer identification.

Note: the communication is insecure, the signalling server can read all content exchanged.

Hence, to replace the signalling server with Waku, we would need to implement a protocol that does just that.

It could be worth considering designing a more generic protocol that enables two peers to communicate with each other, using peer ids as a mean of peer identification. The proposed name is Waku Socket, name proposals are welcome.

Once such protocol is designed and implemented, then we would need to implement it as a libp2p transport to use it for WebRTC handshake. For that, we need to implement listen and dial APIs that take a multiaddr as a listening address and a dial address. As we would be listening and dialing over Waku, we would need to come up with a multiaddr format as part of this generic Waku Socket protocol. .e.g /waku/<pubsub topic>. It may make sense to append a webrtc flag to specify that we are listening for webrtc handshake events, e.g. /waku/<pubsub topic>/p2p-webrtc-star.

Tracked with https://github.com/status-im/js-waku/issues/681

nwaku requirements

In summary, no change requirement for nwaku has emerged from this exploration. It should be possible to do WebRTC handshake over Waku with nwaku as a service node, as it is.

Moreover, so far, no party has expressed an immediate need for WebRTC. Hence, I believe we can take the "long" route of designing a protocol that enables handshakes done in a decentralized manner.

However, if a party has an immediate need for browser-to-browser WebRTC, it should be straightforward to include signalling server logic in nwaku. The one challenge would be to make both browser nodes connect to the same nwaku node to make the handshake happen. This should not be too difficult as it would simply require the Browser nodes to agree on a service node to connect to.

[D4nte]

Icing this until #681 is done.

[D4nte]

A correction on WebRTC direct.

It was previously stated that no ssl certificate is needed for a webrtc direct connection. This statement was incorrect and due to an improper testing environment:

1. webrtc-direct listener is listening on a remote node
2. webrtc-direct dialer is embedded in a local index.html file, opened in a local browser.

Note: WebSocket Secure must be used in a secure context, ie, https. It is not possible to do a clear WebSocket connection within a secure context (https served page). Modern browser failed silently to the user (errors are displayed in console).

When doing the test with the environment above, the index.html is served in an insecure manner, locally. When the context is local, the context is not secure, hence, non secure WebSocket connections can be done. Which led the tester (me) to believe that no certificate is needed to use WebRTC Direct.

As this is opposite to what the libp2p folks have stated I have performed the test again, this time:

1. webrtc-direct listener is listening on a remote node
2. webrtc-direct dialer is server on a remote node in a secure context (https).

In this setup, we can see in the console the following error message:

Blocked loading mixed active content “http://waku.fryorcraken.xyz:9090/?signal=...”

Indeed, it attempts to perform an insecure connection http in the secure context https and the browser block it.

So no, webrtc-direct is not a great candidate as it does not enable a connection to a remote node without an SSL certificate being setup...

Except if we use a Waku Socket like process where the SDP (signal data payload) is being exchanged over an existing Waku connection, remove the necessity of a wss connection to the target node. E.g.:

• Alice: Browser
• Bob: nwaku with wss
• Carole: nwaku, no wss, with WebRTC direct

sequenceDiagram
	  participant A as Alice
	  participant B as Bob
	  participant C as Carole
	  A-->>B: WSS Connection
	  A->>B: SDP over Waku Relay
	  B->>C: SDP over Waku Relay
	  C->>B: SDP answer over Waku Relay
	  B->>A: SDP answer over Waku Relay
	  A-->>C: WebRTC Connection

However, while an option. This seems to be cumbersome one. It could be explored once handshake over Waku for p2p WebRTC is done.

WebTransport looks like a better option to avoid operating having to setup CA signed certificates: https://github.com/libp2p/specs/pull/404/files

[oskarth]

As far as services being provided by the service network goes, having an nwaku node with optional letsencrypt certificate seems doable, especially with a concentration of 1:100 or 1:1000.

Assuming such a node exist, and there's a way for browser nodes to agree on which node to use (for now), are there any blockers for WebRTC usage in browser for e.g. a direct browser comm protocol?

[D4nte]

As far as services being provided by the service network goes, having an nwaku node with optional letsencrypt certificate seems doable, especially with a concentration of 1:100 or 1:1000.

Assuming such a node exist, and there's a way for browser nodes to agree on which node to use (for now), are there any blockers for WebRTC usage in browser for e.g. a direct browser comm protocol?

@oskarth. AFAIK, there are no blockers to use WebRTC with Waku to enable direct browser-to-browser communication. The current known unknowns are:

1. designing a protocol for signaling and implementing
2. whether NAT/firewall are an issue and how to deal with it.

Further exploration on 2. is necessary, I'll create an issue to track.

[weboko]

Following up on the example of Noise+WebRTC usage I'd like to post it's progress here:

State of the example: work in progress

What's done:

• By using js-noise users can establish secure communication channel.
• This channel is used to exchange offer/answer to initiate direct WebRTC connection.

What should be done:

• STUN server: in order not to loose benefits of peer-to-peer protocols used underneath we should find a way to be able to retrieve coordinates of a user to build offer/answer by not involving one STUN server for it;
• TURN server: similarly to prev point we should be able to cover a need to create WebRTC connection for users who are behind secure NAT and are not able to communicated just as it is.

Additional reading that explains why STUN/TURN is not easily removable from the equation: https://github.com/libp2p/specs/pull/497/files#diff-2cb0b0dcc282bd123b21f5a0610e8a01b516fc453b95c655cf7e16734f2f7b11R48-R53

[fryorcraken]

This issue is a bit cluttered and enabled us to learn further about WebRTC.

Next steps tracked in https://github.com/waku-org/js-waku/issues/1181