session, staking: introduce static limit on the number of validators

[andresilva]

We need to introduce a static value for the maximum number of validators allowed in staking and session, we can then use this limit to replace a couple of storage entries with length-bounded data structures, e.g. DisabledValidators, Invulnerables, OffendingValidators can use BoundedVec.

See: https://github.com/paritytech/substrate/pull/9448#discussion_r693232481

[ShubhamPalriwala]

Hey, so we can have the DisabledValidators, Invulnerables, and OffendingValidators use the BoundedVec. Apart from that. how are we having an upper limit, is it static or dynamic? And how do we want to validate it?

[burdges]

I'm confused by "static" here. You want a global limit no chain ever exceeds? Or you want some more authoritative governance controlled limit?

We believe no substrate chain will ever exceed 3072 validators because we believe the OmniLedger sharding approach kicks in before 1500 validators, possibly before 768, and someone might require a factor of two so the set could be split. We envision doing this with the polkado relay chain eventually. We might require better randomness for this, but that's preferable to running gossip on more than 3072 validators.

[andresilva]

We want a static limit that can only be changed by a runtime upgrade, similar to what exists here: https://github.com/paritytech/substrate/blob/master/frame/babe/src/lib.rs#L176-L178, so that we can use this limit to use size-bounded collections like here: https://github.com/paritytech/substrate/blob/master/frame/babe/src/lib.rs#L198-L205.

For example this vector in session pallet can currently grow indefinitely: https://github.com/paritytech/substrate/blob/master/frame/session/src/lib.rs#L523-L530

cc @kianenigma ?

[burdges]

How do the size-bounded collections help? It's weight computations?

I suppose 3072 fits nicely if you do not matter a factor of 2x or 3x but likely going that high requires other runtime upgrades, so it could be set lower if required.

[Doordashcon]

The static limit can be set by entities adding staking and session pallets to their runtimes, for the node I believe 3072 is acceptable?

Then add this bound to relevant storage items.

Might be an over simplification on my part.

[kianenigma]

I've made a draft of the proper way to do this in https://github.com/paritytech/substrate/tree/kiz-properly-bound-staking-validators

@Doordashcon this is how you should go about doing https://github.com/paritytech/substrate/pull/12125. Nonetheless, as noted in https://github.com/paritytech/substrate/pull/11967#pullrequestreview-1079629888, this might be a little bit of an involved issue since it touches multiple pallets and perhaps you should work with someone from our team to do this.

I will try and create a new tracking issue for bounding the staking pallet, and devise a plan there. Hopefully we can come up with some better small steps for you to contribute through.

[kianenigma]

Replaced with https://github.com/paritytech/polkadot-sdk/issues/255