lodestar icon indicating copy to clipboard operation
lodestar copied to clipboard

Increased attester duties response times since Deneb

Open nflaig opened this issue 4 months ago • 1 comments

Describe the bug

From Holesky validator client metrics, the getAttesterDuties response time has gotten a lot more spiky since Deneb

image

There might also be a slight increase in signature times (needs further observations)

image

Expected behavior

Same performance / response times as in Capella

Steps to reproduce

Run Lodestar vc <> bn on Holesky

Additional context

More details / metrics can be found on discord

Operating system

Linux

Lodestar version or commit hash

v1.15.0

nflaig avatar Feb 09 '24 10:02 nflaig

I went through all the Lodestar dashboards to see if I could find any hints at what this could be caused by.

Here are the ones that stood out, hope it helps!

increase(lodestar_stfn_state_cloned_count_count[1h])

image

--> the states seems to be cloned 2x more often post-Deneb.

image image image image

eth2353 avatar Feb 09 '24 11:02 eth2353

looks like this time has been stable over time, this is in the last 30 days

Screenshot 2024-03-25 at 15 47 52

on mainnet node it's quite good

Screenshot 2024-03-25 at 15 50 27

twoeths avatar Mar 25 '24 08:03 twoeths

Performance looking good on our end nowadays too, running 1.17.0 though (issue was reported by us on 1.15.0). Feel free to close this Tuyen / team, seems to no longer be an issue! 🙂

eth2353 avatar Mar 25 '24 18:03 eth2353

Looks good on my mainnet node as well, holesky also not too bad, closing this for now. Thanks @eth2353 for the metrics :pray:

nflaig avatar Mar 25 '24 19:03 nflaig