substrate icon indicating copy to clipboard operation
substrate copied to clipboard

Implements dynamic number of nominators

Open gpestana opened this issue 2 years ago • 11 comments

This PR refactors the implementation of the ElectionDataProvider bounds and implements a dynamic nomination quota per voter, based on the account's balance and an arbitrary curve.

Instead of a fixed MAX_NOMINATIONS per nominator, the ElectionDataProvider uses an implementor of trait NominationsQuota<Balance> to calculate the max votes per nominator. Since the number of votes per nominator is dynamic, the election data provider (pallet staking) keeps track of the size of the snapshot to ensure that the final snapshot size is bounded, even when the number of votes per voter is calculated on the fly. This PR implements an ElectionSizeTracker to achieve this goal.

The FixedNominationsQuota<const MAX: u32> struct implements the nomination quota trait and can be used to set the maximum number of nominations per nominator to 16, which is the current behaviour.

Lazy nomination quota checks: Note that if a nominator's quota decreases after the nomination has been done, all the nominations will be used. The nominations quota limits is only enacted when an account performs a new nomination, in which case, if the number of nomination exceeds the current nominator quota, the fn nominate extrinsic fails with TooManyTargets.

Notable changes:

Trait NominationsQuota

An implementor of NominationsQuota returns the maximum number of votes per nominator. FixedNominationsQuota implements an instance of a fixed number of nominations, similar to the current implementation when set to 16.

ElectionsBounds and DataProviderBounds

The MaxElectableTargets and TargetsBound are replaced by ElectionBounds. ElectionsBounds defines the count and size limits of both voters and targets for an election.

The struct ElectionBounds sets the upper bounds in size (MB) and number of voters/targets to request from the election data provider. The bounds are defined over two axis: number of elements and size (in MB) of the snapshot that is built from the election result.

Struct ElectionSizeTracker

Keeps track of the amount of data (in size and number of elements) that an election data provider is preparing in get_npos_*. The election data provider uses an instance of ElectionSizeTracker to know when the number of voters exceeded the bounds in terms of size and return the election data to EPM.

It is important to keep track of the size in MB of the final snapshot since the number of nominations per nominator (aka edges in the snapshot) is now dynamic and may be large.

Changes to the ElectionDataProvider interface

Both fn ElectionDataProvider::election_voters and fn ElectionDataProvider::electable_targets take as an input an instance of DataProviderBounds, instead of an optional max in the of number of returned elements.

New events

  • SnapshotVotersSizeExceeded: emitted when the election snapshot was exhausted due to the size limit.

Runtime API

This PR also adds a new runtime API to allow clients to query the nominations quota for a given balance.


polkadot companion: https://github.com/paritytech/polkadot/pull/6807

Related to and reviving this PR and discussion https://github.com/paritytech/substrate/pull/10340

Helps https://github.com/paritytech/substrate/issues/13069 Closes https://github.com/paritytech/substrate/issues/12968

gpestana avatar Dec 19 '22 11:12 gpestana

bot fmt

gpestana avatar Mar 01 '23 11:03 gpestana

bot rebase

gpestana avatar Mar 01 '23 19:03 gpestana

bot rebase

rossbulat avatar Mar 06 '23 04:03 rossbulat

The only concern I annotated is with the potential repetition of the SnapshotVotersSizeExceeded event, that could maybe undesirably populate a block with such events if we have many over-exceeded nominators.

Do you mean NominationsQuotaExceeded? This event is the one that may be emitted multiple times per election round (imagine if X voters had quota exceeded when doing the election round, the event will be emitted X times) and there is high chances it will be once we start tweaking the nomination quota curve. The SnapshotVotersSizeExceeded may only happen once per election.

gpestana avatar Mar 06 '23 07:03 gpestana

Yes sorry, I meant NominationsQuotaExceeded.

rossbulat avatar Mar 06 '23 07:03 rossbulat

This pull request has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/polkadot-release-analysis-v0-9-39/2277/1

Polkadot-Forum avatar Mar 09 '23 19:03 Polkadot-Forum

bot fmt

gpestana avatar Mar 16 '23 13:03 gpestana

bot cancel 70-eb05f2e6-1a96-4cf4-841f-3805c9c9031a

gpestana avatar Mar 16 '23 13:03 gpestana

bot bench $ pallet dev pallet-election-provider-multi-phase

gpestana avatar Mar 25 '23 16:03 gpestana

bot rebase

gpestana avatar May 17 '23 07:05 gpestana

bot rebase

gpestana avatar Jun 02 '23 13:06 gpestana

We will look into empirical data and suggest a curve by about next week.

FatemeShirazi avatar Jun 16 '23 10:06 FatemeShirazi

bot fmt

gpestana avatar Aug 05 '23 17:08 gpestana

bot rebase

gpestana avatar Aug 07 '23 13:08 gpestana

bot rebase

gpestana avatar Aug 08 '23 10:08 gpestana

bot fmt

gpestana avatar Aug 08 '23 18:08 gpestana

bot clean

gpestana avatar Aug 08 '23 18:08 gpestana

bot rebase

gpestana avatar Aug 09 '23 12:08 gpestana

Rebased

bot merge

gpestana avatar Aug 10 '23 07:08 gpestana

cargo test --release -p pallet-staking fails for me after this PR. Apparently some tests are supposed to panic with debug_assert which only works in debug config.

arkpar avatar Aug 12 '23 12:08 arkpar