vega [Bug]: Investigate 5s blocks

[Bug]: Investigate 5s blocks

Open edd opened this issue 10 months ago • 1 comments

Problem encountered

There are periodic slow blocks on this network. These correlate with larger transaction counts - 100+ txs. Current benchmarks are not flagging up anything that should cause slowness in these blocks

Observed behaviour

It's not all that frequent, but it is common
They do not happen on a regular schedule
They are distributed across validators (see spreadsheet screenshot below)

The makeup of the larger blocks is relatively ordinary:

100-ish batch transactions
- Most contain a cancel all in market order
- Place 3-15 orders
The remaining TXs are generally Chain Events for oracle data

Expected behaviour

Block time does not increase 5x when the tx count is only 3x

Steps to reproduce

1. Observe Grafana for slow blocks
2. Hover over the graph and get a timestamp
3. Head over to explorer and find the block for that timestamp
4. Inspect block contents