ethereum
triedb/pathdb, eth: use double-buffer mechanism in pathdb
Previously, PathDB used a single buffer to aggregate database writes, which needed to be flushed atomically. However, flushing large amounts of data (e.g., 256MB) caused significant overhead, often blocking the system for around 3 seconds during the flush.
To mitigate this overhead and reduce performance spikes, a double-buffer mechanism is introduced. When the active buffer fills up, it is marked as frozen and a background flushing process is triggered. Meanwhile, a new buffer is allocated for incoming writes, allowing operations to continue uninterrupted.
This approach reduces system blocking times and provides flexibility in adjusting buffer parameters for improved performance.
TODO:
- [ ] release the content in the frozen buffer after flushing
For sure, it’s not a please-merge-it pull request, it will be twisted a bit and have a full performance impact inspection
Thanks and Best regards Gary rong
Martin HS @.***>于2024年9月23日 周一下午2:49写道：
@.**** approved this pull request.
LGTM, would be interesting to see some performance-charts. This PR needs some runtime before merging, IMO
— Reply to this email directly, view it on GitHub https://github.com/ethereum/go-ethereum/pull/30464#pullrequestreview-2321245656, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNO6OOSVOX3WRFPVBR6AJ3ZX62XNAVCNFSM6AAAAABOPFBYCCVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDGMRRGI2DKNRVGY . You are receiving this because you authored the thread.Message ID: @.***>
The triedb commit is improved as expected, but the EVM execution is constantly slower, with unknown reasons.
Referenced #28471 ？
I have the same question, is this PR referenced #28471 ？
@joey0612 @waynercheung
No, double-buffering has been part of my design since day one when I first implemented the path database. I even had a prototype a few years ago. However, it hasn’t been merged yet for several reasons: (a) the overall performance improvement was not significant; and (b) after flushing the write buffer into the database, subsequent database reads became 2–3× slower, which in turn slowed down the entire chain progression.
However, with the fix to block prefetcher, the state read has been significantly improved. And I want to pile up this change on top one more time.
https://github.com/cockroachdb/pebble/issues/4109