consensus-specs
consensus-specs copied to clipboard
ethereum
Deprecate mplex
Mplex was a simple stream multiplexer created in the early days of libp2p.
It has since been deprecated and is now potentially becoming a security vulnerability as it is inefficient and unmaintained.
The general recommendations are to use yamux or QUIC.
Lighthouse currently supports mplex and yamux over TCP and QUIC. We are hoping to deprecate mplex and remove support for it in the near future.
We don't want to harm other clients in this process, so we are hoping for some visibility on this deprecation.
Current Progress
| Client | Mplex | Yamux | QUIC |
|---|---|---|---|
| Lighthouse | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| Prysm | :heavy_check_mark: | :heavy_check_mark: | :question: |
| Nimbus | :heavy_check_mark: | :x: | :construction: |
| Teku | :heavy_check_mark: | :x: | :construction: |
| Grandine | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
From what I gather @philknows : Lodestar is ready, but needs to support just yamux and remove mplex at once @nisdas : Prysm is ready @arnetheduck : Nimbus is testing yamux but its not quite ready for production yet @lucassaldanha : Teku has yamux in experimental phase, needs a bit of time to complete it.
I guess it would be nice to set a rough timeframe for Nimbus and Teku to be ready. Would someone like to suggest a rough time frame? We can also target a fork if we want?
From nimbus, our preference would probably be to go straight to QUIC and skip the yamux step, given that yamux is pretty much a dead end anyway (de-facto unmaintained). mplex can hang on for a bit longer given its simplicity and battle-testing so far.
From Teku perspective, as mentioned we still have work to do on yamux so just going straight to QUIC would be preferable for us since it looks like the way forward anyways.
Ok. Looks like consensus is to move straight to QUIC.
I'll keep this issue open and try and make a tally of progress. So we know how close we are to being able to shift.
I am closing this issue because it seems stale. Please, do not hesitate to reopen it if this is a mistake
Ok, reviving this old PR.
It has been over a year since we brought this up. I think this is a reasonable amount of notice time. We are planning to remove mplex support from Lighthouse at the end of the year. cc @jxs
Ok, reviving this old PR.
Nice 👍 it would be good to get a status update from clients.
cc @philknows @kasey @arnetheduck @lucassaldanha @sauliusgrigaitis
Do all clients support QUIC now?
Do all clients support QUIC now?
Still WIP for us. We were working on it, but had to stop due to Fulu. We are planning to finish it ASAP now that Fulu is out of the door (🤞 )
Ditto nimbus, where it needs finishing - since we last wrote here, the quic backend for libp2p is has been completed but its integration into eth2 has not (high prio)
In a testing phase for Lodestar. Needs some dusting up and monitoring for stability now that Fulu is shipped.
The goal is to migrate to QUIC as a primary transport, keeping TCP+Yamux+Noise as a fallback transport stacking because ocassionally middleboxes block UDP traffic.
In terms of sequencing, this is the safest way to conduct the shift:
- All clients support QUIC, selecting it with the highest precedence and falling back to TCP in case of failure.
- All clients support Yamux, at least until Qmux is stabilized. Qmux is a draft IETF standard to create a polyfill enabling apps built against a QUIC API to run on alternative transports like TCP.
- Only then, deprecate mplex.
If we agree on this, @marcopolo or I can submit an alternative PR that also removes spystream from the spec.
keeping TCP+Yamux+Noise as a fallback transport stacking because ocassionally middleboxes block UDP traffic.
My understanding is that UDP support is already a requirement for discv5.
TCP+Yamux+Noise as a fallback transport
TCP+Mplex+Noise - we never made the move to yamux formally so mplex is the only supported fallback option in clients - the goal being to skip yamux entirely since it's a known dead end.
because ocassionally middleboxes block UDP traffic.
Btw, we already have a de-facto hard requirement on UDP working (discv5), so dropping TCP is a strictly an increase in the ability for nodes to connect, removing TCP from the mix.
TCP+Yamux+Noise as a fallback transport
TCP+Mplex+Noise - we never made the move to yamux formally so mplex is the only supported fallback option in clients - the goal being to skip yamux entirely since it's a known dead end.
I agree with that. There is no point in my opinion in spending engineering time with having an efficient Yamux implementation which will essentially act as a fallback when we have a battle tested TCP + Mplex in all clients. Maybe we can deprecate Yamux instead?
@StefanBratanov, I'm not convinced we need a TCP fallback, but if we did we do not want to settle on mplex for the reasons listed here: https://github.com/libp2p/specs/pull/661/files
I'm aware talk is cheap (especially cheaper than engineering effort) but I think having multiple robust networking paths (both UDP and TCP) is completely aligned with Ethereum's core principles (Durable and Reliable).
The issue with TCP+Noise+Mplex is that the Mplex spec doesn't require flow control (and it's technically deprecated by libp2p)
Notably, mplex does not provide flow control, a feature which is now considered critical for a stream multiplexer.
Therefore I agree with @AgeManning and @raulk, that the consensus-spec should recommend support for (in order of preference)
- QUIC
- TCP+Noise+Yamux
- TCP+Noise+Mplex (only while teams build out Yamux support)
And looking out further, maybe we deprecate the TCP+Noise+Yamux stack once every language has a Qmux implementation.
But maintaining reliable network paths on both TCP and UDP should be required, in my opinion.
The whole ethos of libp2p is that it's modular and can support a wide range of protocols simultaneously. So why not leverage that benefit?
mplex does not provide flow controw
not mplex for the reasons listed here
This is mostly a theoretical issue, ie the marginal benefit to ethereum of this feature is close to none due to how the protocol is structured - whatever benefit may be found is provided by quic already (and quic has other advantages).
can support
The ethereum spec only needs to specify minimal support for interoperability - clients are free to support anything beyond that if they so desire (and can specify that in their own documentation).
Practically, yamux is just as dead as mplex in terms of upstream support, so it doesn't contribute anything meaningful really that quic doesn't provide already - it is however a lot more complex than mplex (and because it was never required, several clients don't have hardened / performant implementations). Having it in this spec means the security surface of the protocol increases without any real benefit.