Implement `selections` Beacon API endpoints to support DVT middleware

[chong-he]

Issue Addressed

• #6610

Proposed Changes

• [x] Add beacon_committee_selections endpoint
• [x] Test beacon committee aggregator and confirmed working
• [x] Add sync_committee_selections endpoint
• [x] Test sync committee aggregator and confirmed working

Additional Info

Thank you @michaelsproul and @macladson for the help and guidance

[chong-he]

With the commit 8b2f058 where VC sends partial signatures in parallel for all validators for a slot, I believe the aggregated attestations are successful (during testing) now, thank you @michaelsproul !

Lighthouse VC logs:

{"log":"Mar 20 06:36:24.046 INFO Successfully published attestation      type: aggregated, slot: 37, committee_index: 0, head_block: 0xdb186315afecb3d1134d03eb2d5b04b83a21ede56be6563e3bff304b103bf4b6, signatures: 24, aggregator: 58, service: attestation\n","stream":"stderr","time":"2025-03-20T06:36:24.04690072Z"}

for every slot now

Charon logs:

{"log":"06:36:24.074 INFO bcast      Successfully submitted attestation aggregations to beacon node {\"delay\": \"8.074945477s\", \"duty\": \"37/aggregator\", \"peer\": \"twinkling-pen\", \"protocol\": \"/charon/parsigex/2.0.0\"}\n","stream":"stderr","time":"2025-03-20T06:36:24.074999639Z"}

for every slot now too

@KaloyanTanev could you give it a try and confirm that the beacon committee selection endpoint is working as intended?

Thanks

[mergify[bot]]

This pull request has merge conflicts. Could you please resolve them @chong-he? 🙏

[KaloyanTanev]

Tested on local Kurtosis chain and it works!

[chong-he]

Tested on local Kurtosis chain and it works!

Thanks for confirming, I am now working on the sync committee selection endpoint

[michaelsproul]

Could log the selection_proof and aggregator_index when posting the SignedContributionAndProof here:

https://github.com/sigp/lighthouse/blob/b21451fc114810e4e36777ec6b3c3d5072e15de9/validator_client/validator_services/src/sync_committee_service.rs#L398

This selection proof should be the same as the full proof.

[michaelsproul]

One more comment from debugging, the signature that is failing for sync committee selection proofs is the aggregated selection proof, here:

https://github.com/ObolNetwork/charon/blob/22ce8f0ef250cef7d4942e14ba0571f8fe357584/core/validatorapi/validatorapi.go#L928

[chong-he]

As per ef90462, testing shows that:

Lighthouse VC:

{"log":"Mar 27 13:05:18.097 INFO Successfully published sync contributions, slot: 43, num_signers: 128, beacon_block_root: 0x965f3d26d40f79949eff6ec2a7e5e6a4f59984beb78173b0ea98100774981f7b, subnet: 2, service: sync_committee\n","stream":"stderr","time":"2025-03-27T13:05:18.09860353Z"}

Charon:

{"log":"13:05:18.129 INFO bcast      Successfully submitted sync committee contributions to beacon node {\"delay\": \"8.128996601s\", \"duty\": \"43/sync_contribution\", \"peer\": \"calm-street\", \"protocol\": \"/charon/parsigex/2.0.0\"}\n","stream":"stderr","time":"2025-03-27T13:05:18.129227056Z"}

for every slot. This should imply that the sync_committee_selection endpoint is working. Waiting confirmation from Obol.

Edit: confirmed by @KaloyanTanev that Sync committee selection endpoint is working as intended, thanks for testing it. I will clean up the code a bit to get the PR ready for review

[dknopik]

Hello @chong-he and @michaelsproul,

I found this PR and am interested in the changes in fill_in_selection_proofs as Anchor's selection proofs do not work with the current state from unstable, but this PR fixes it partially. Unfortunately, there are some differences between --distributed mode and the behaviour required by Anchor, so I propose we make this PR a bit more general in order to accommodate everyone's needs.

To illustrate the differences:

	VC	VC with --distributed	Anchor
Look ahead	8	1	0 (current slot only)*
Run at x seconds into slot	6	6	0*
Use middleware endpoints	No	Yes	No
Sign in parallel	No	Yes	Yes**

*: Alternatively, this can be expressed as lookahead of 1, ran 12 seconds into the slot. **: Ideally, results should handled as they come in, since some signatures may fail late. In other words, await a stream instead of using join_all.

We could bundle these configuration options in a struct (e.g. SelectionProofConfig) and pass it to the duties service. This way, Lighthouse can pass in the required values based on whether --distributed is specified, and Anchor can pass in its values.

Looking forward to hear your thoughts. Let me know if you have any questions.

[mergify[bot]]

This pull request has merge conflicts. Could you please resolve them @chong-he? 🙏

[chong-he]

Thanks! I've added a few notes.

Thank you so much for the review and help. All revised accordingly, let me know if further changes required.

[dknopik]

I am still confused about the slot - 1. But when removing it, this PR looks good to me from the perspective of the functionality needed by Anchor. @michaelsproul do you know why that is there? By my understanding of the Altair spec, the proof for the previous slot is not at all relevant for the current slot.

[mergify[bot]]

Some required checks have failed. Could you please take a look @chong-he? 🙏