ssv icon indicating copy to clipboard operation
ssv copied to clipboard

implement proposal scoring and best selection

Open vyzo opened this issue 1 week ago • 10 comments

This implements proposal selection from multiple beacons.

The behavior changes from selecting the first received one, as it has been observed in practice that it is not always the most profitable. A small time is allotted for selection, during which proposals are collected to select the one with the best score. If no valid proposal is collected at the allotted time, it falls back to the first received to avoid increasing latency too much.

The proposal score is the one used in vouch -- see https://github.com/attestantio/vouch/blob/9e03b5be9ce3c4553244b628f3d662561c1d20bc/strategies/beaconblockproposal/best/score.go#L25

The timeout strategy is also similar to vouch.

Remaining Work

  • [ ] unit test; an LLM can be coaxed to do this.
  • [ ] test in our staging environment

vyzo avatar Dec 18 '25 17:12 vyzo

Codecov Report

:x: Patch coverage is 75.30864% with 20 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 57.2%. Comparing base (8808f67) to head (982944f).

Files with missing lines Patch % Lines
beacon/goclient/proposer.go 75.3% 14 Missing and 3 partials :warning:
observability/traces/trace.go 25.0% 3 Missing :warning:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

codecov[bot] avatar Dec 18 '25 17:12 codecov[bot]

I added it to the options in https://github.com/ssvlabs/ssv/pull/2631/commits/ae96a066715817f17e2201798477cdcb7dbf2ddb and set defaults for 1s and 2s respectively.

vyzo avatar Dec 19 '25 11:12 vyzo

I agree, but i am not sure we need to solve it in this pr; we should probably make a follow up issue to get this right.

vyzo avatar Dec 19 '25 12:12 vyzo

I agree, but i am not sure we need to solve it in this pr; we should probably make a follow up issue to get this right.

Yup, we can do a follow-up PR for that, just seems kinda logical to me to group these "features" together (even though nothing really makes it "necessary" to do so).

iurii-ssv avatar Dec 19 '25 13:12 iurii-ssv

lets get some logs first and then we can decide.

vyzo avatar Dec 19 '25 13:12 vyzo

Hey, nice work on this! I've been thinking about how this interacts with proposer timing games.

Question about the tradeoff:

I'm trying to understand the expected gain from the collection window vs. the cost of reduced builder time. With the ~1.6s soft timeout, we're effectively reducing the proposerDelay budget available for block building.

For typical setups (shared MEV-boost or same relays across MEV-boosts), I'd expect all beacon clients to return essentially the same block from the same winning relay bid. In that case, time seems like the primary value driver rather than selection between near-identical proposals.

An alternative approach:

What about differentiating based on what comes back first?

  • First response is MEV (blinded): Take it immediately, maximize builder time. The proposals from other clients will likely be the same anyway.
  • First response is vanilla: Wait briefly (soft timeout) in case another client returns an MEV block - vanilla first might indicate one client's MEV-boost had issues.

This way, we get the best of both worlds - maximum builder time for MEV while still having a safety net to catch MEV blocks when the first response happens to be vanilla.

liorrutenberg avatar Dec 21 '25 15:12 liorrutenberg

@andrew-blox also suggested checking another optimization: request vanilla blocks in parallel while waiting for MEV. That way if MEV fails or times out, we already have vanilla ready without the extra round-trip latency.

liorrutenberg avatar Dec 21 '25 15:12 liorrutenberg

I like the idea of taking MEV block immediately if it arrives within the soft timeout. I will add it tomorrow.

The requests are all parallel, so we dont have to do anything more.

vyzo avatar Dec 21 '25 15:12 vyzo

@vyzo I looked through the changes on the stage environment and it seems we manage to propose and submit most proposals with MEV but we seem to fail quite a high percentage also, I see the proposals suffer some sort of 3 second "hang" on submittions, can't put my finger on what exactly the reason for this. I also looked at the waited out proposer delay and saw delay was 0ms on every proposal, not sure if it's an issue but it seems we might be missing something

Roy-blox avatar Dec 21 '25 16:12 Roy-blox

ok, these are findings we need to address.

vyzo avatar Dec 21 '25 16:12 vyzo