Mux: Single peer performance for Leios

[coot]

Why

See https://github.com/input-output-hk/ouroboros-leios/issues/544#issue-3398692866

We need to prioritize Praos > Leios (and slightly less important: fresh Leios > stale Leios). With near certainty, Praos and Leios traffic will be implemented using two separate mini protocols. Hence, we want to limit the impact one mini-protocol can have on the other and further explore options how to improve single peer performance.

What

• [ ] #5261 will allow us to independently of the consensus demo validate early implementations, later on it can serve as a basis for writing proper tests, and eventually give us data points to initially configure mux with Leios capabilities, which later can be validated using the consensus Leios demo.
• [ ] #5262 we will explore prioritisation of egress queues to support Leios design requirement to prioritise Praos traffic over Leios traffic. It will be early-validated using #5261 and later on using the consensus Leios demo.
• [ ] #5263 we will explore dynamic egress rate limiting. It will be early-validated using #5261 and later on using the consensus Leios demo.
• [ ] For prioritisation of fresh Leios traffic over stale Leios traffic, we plan to do server-side reordering. An early idea was explored in input-output-hk/ouroboros-leios#638.

[ch1bo]

@coot Should we make it more clear this is explorative work and not necessarily a production-grade implementation of the solution ideas? i.e. a "spike" or "experiment" is fine for Leios at this stage

[coot]

We can be a bit more strict. Usually, exploratory work ends up as production code :grin: (that's what happened to cardano-ledger when we rewrote cardano 😄). We can initially save on testing, but implementations should be proper. It's different from the consensus demo, which is much larger in scope and thus has many more opportunities for shortcuts, which we don't want to have in production code. Unlike in the consensus demo, the changes here are localised to a single component, so it's much easier for us to do it properly.