go-spacemesh
go-spacemesh copied to clipboard
fetch: add streaming mode
Motivation
Excess buffering during sync causes unneeded GC pressure and sometimes leads to out-of-memory errors.
Description
Added streaming mode for the fetcher (feature-gated).
In streaming mode, no extra buffering is used for fetch server and client. Fetch client still uses some big slices such as list of ATX IDs in an epoch, but that's to be improved in a followup.
Streaming mode is enabled via fetch.streaming
boolean config key, which defaults to false.
Test Plan
Verified sync on the testnet.
Codecov Report
Attention: Patch coverage is 79.42446%
with 143 lines
in your changes are missing coverage. Please review.
Project coverage is 80.0%. Comparing base (
b09651a
) to head (da777b2
). Report is 3 commits behind head on develop.
Additional details and impacted files
@@ Coverage Diff @@
## develop #5562 +/- ##
=========================================
+ Coverage 79.8% 80.0% +0.2%
=========================================
Files 279 279
Lines 28549 29009 +460
=========================================
+ Hits 22789 23223 +434
+ Misses 4186 4184 -2
- Partials 1574 1602 +28
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
(almost forgot) Will further update this PR to move some of the codec stuff to go-scale
, otherwise good for review
bors try
bors try
bors merge
bors cancel
Canceled.
bors merge