beacon-APIs icon indicating copy to clipboard operation
beacon-APIs copied to clipboard

Add validators v2.

Open mcdee opened this issue 1 year ago • 14 comments

This PR adds a version 2 of the validators API.

The requirement comes from the number of entries in the Validators array in state on mainnet. At time of writing there are over 1.4MM entries, and no plans to reduce this number (there are some options that could reduce the number of active validators, but this doesn't help the array itself). The JSON returned by v1 of this response is over 600MB in size, which has knock-on effects for both speed of delivery (it takes approximately 5s to send this much data over a 1GB link) and decoding time. As such, a version of this endpoint that supports SSZ would be useful.

The changes in this endpoint from v1 are:

  • allow SSZ as a return encoding type
  • remove the status item from the per-validator data
  • provide the consensus version in the header and JSON metadata

status is a text string in the JSON response for v1 of this call, so does not have a canonical mapping to SSZ. Instead of removing it, other options are to define a mapping similar to an enum, or to provide it as a byte array. Both come with added complexity or increased data, and the information that it provides can be calculated from the data that is already provided in the validator struct (plus an epoch).

Some open questions around this endpoint:

  • should we add an epoch metadata field in the returned data to allow recreation of validator states for when the state identifier used as part of this call is non-numeric (head, finalized etc.)?
  • should we remove the balance value for each validator? This value is available from the validator_balances endpoint, does it make sense to still duplicate it here?
  • could we realistically remove the index value for each validator? The big benefit of doing this would be that the data returned could be a simple array of Validator objects, which would remove the need for custom structures when decoding SSZ (and, indeed, JSON). However, there is a significant downside here in that the index information would no longer be easy to obtain. Countering that, should we be promoting the use of indices anywhere in the API when it's possible that index reuse will become a thing?
  • if we do not return status, should we still allow filtering the requested values by status?

mcdee avatar May 29 '24 20:05 mcdee