nearcore icon indicating copy to clipboard operation
nearcore copied to clipboard

[ProjectTracking]: Stateless Validation benchmarking

Open staffik opened this issue 1 year ago • 0 comments

Context

Stateless Validation tracking issue.

Goal

Stateless Validation should be benchmarked and analyzed in a presentable format.

Motivation

  • Performance optimization and stability enhancement should be in place based on incoming data.
  • Bowen Wang plans to share findings from Stateless validation StatelessNet during EthDenver (Feb 29th).
  • We need enough confidence on resilience, scalability, reliability of the network before completing StatelessNet (March 31st).

Tasks

  • [x] Prepare and share a plan for benchmarking. Zulip
  • [x] Run locust scenario against StatelessNet, possibly finding issues. Zulip
  • [x] Run locust scenario against adversenet, comparing current Mainnet binary with StatelessNet binary.
  • (ideally due February 15th) In-memory trie is ready.
  • [x] Run the locust scenarios again with in-memory trie enabled (against StatelessNet and adversenet).
  • [x] (due February 21st) Compare data for 3 binary versions: current Mainnet, Stateless Validation with / without in-memory trie and write up a report for EthDenver.
  • (February 29th) EthDenver
  • [ ] (March 15th) Check-in, discuss if we have enough confidence in the network.
  • [ ] (March 31st) Wrap up the StatelessNet test and share the final report.

Prerequisites (ideally due February 15th)

  • https://github.com/near/nearcore/issues/10462
  • https://github.com/near/nearcore/issues/10563
  • https://github.com/near/nearcore/issues/10564

staffik avatar Feb 01 '24 09:02 staffik