optimization: reduce snapshot creation time
Describe your changes and provide context
Implement pipeline-based parallel writing to significantly improve snapshot rewrite performance.
- Prefech files into page cache before snapshot creation
- Pipeline parallel writes: Decouple traversal and I/O via 3 background workers
- Large IO/channel buffer: 10k → 500k to reduce blocking and syscalls
- Priority EVM strategy: Write largest tree first to avoid I/O contention
- Skip too frequent snapshot during catch-up state
Testing performed to validate your change
- On a 250GB RAM machine with 250 MiB/s GT3 disk, snapshot creation time reduced from >3 hours to ~20 minutes.
The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).
| Build | Format | Lint | Breaking | Updated (UTC) |
|---|---|---|---|---|
✅ passed | ✅ passed | ✅ passed | ✅ passed | Nov 18, 2025, 10:47 AM |
Codecov Report
:x: Patch coverage is 66.36971% with 151 lines in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 43.72%. Comparing base (cfd70df) to head (52cb9f8).
:warning: Report is 1 commits behind head on main.
:x: Your patch status has failed because the patch coverage (66.36%) is below the target coverage (70.00%). You can increase the patch coverage or adjust the target coverage.
Additional details and impacted files
@@ Coverage Diff @@
## main #2537 +/- ##
==========================================
+ Coverage 42.51% 43.72% +1.20%
==========================================
Files 1271 1626 +355
Lines 104353 139858 +35505
==========================================
+ Hits 44370 61148 +16778
- Misses 56070 73177 +17107
- Partials 3913 5533 +1620
| Flag | Coverage Δ | |
|---|---|---|
| sei-chain | 44.36% <30.76%> (-0.01%) |
:arrow_down: |
| sei-cosmos | 40.98% <0.00%> (+<0.01%) |
:arrow_up: |
| sei-db | 44.83% <67.89%> (+1.53%) |
:arrow_up: |
| sei-tendermint | 47.00% <ø> (?) |
|
| sei-wasmd | 42.74% <ø> (+0.01%) |
:arrow_up: |
| sei-wasmvm | 39.65% <ø> (ø) |
Flags with carried forward coverage won't be shown. Click here to find out more.
| Files with missing lines | Coverage Δ | |
|---|---|---|
| app/seidb.go | 82.69% <100.00%> (+0.33%) |
:arrow_up: |
| app/test_helpers.go | 31.56% <100.00%> (+0.07%) |
:arrow_up: |
| sei-db/sc/memiavl/opts.go | 100.00% <100.00%> (ø) |
|
| sei-db/ss/pebbledb/iterator.go | 54.20% <100.00%> (ø) |
|
| sei-db/config/config.go | 0.00% <0.00%> (ø) |
|
| sei-db/sc/store.go | 0.00% <0.00%> (ø) |
|
| sei-cosmos/baseapp/baseapp.go | 65.16% <0.00%> (-0.12%) |
:arrow_down: |
| sei-db/sc/memiavl/multitree.go | 78.92% <92.59%> (+2.26%) |
:arrow_up: |
| app/app.go | 73.01% <10.00%> (-0.23%) |
:arrow_down: |
| sei-db/sc/memiavl/import.go | 72.72% <70.00%> (+0.75%) |
:arrow_up: |
| ... and 2 more |
:rocket: New features to boost your workflow:
- :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
- :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.