flow-go icon indicating copy to clipboard operation
flow-go copied to clipboard

[State Sync] Experiment with using pebble as the execution datastore db

Open UlyanaAndrukhiv opened this issue 1 year ago • 1 comments
trafficstars

Close #6017

Context

In this pull request:

  • Added a CLI flag that controls which db type to use.
  • Added skeleton of StorageDB interface and badger and pebble implementations.
  • Refactored execution data tracker and execution data pruning mechanism.

UlyanaAndrukhiv avatar Jul 04 '24 17:07 UlyanaAndrukhiv

Codecov Report

Attention: Patch coverage is 8.60215% with 170 lines in your changes missing coverage. Please review.

Project coverage is 41.42%. Comparing base (60e14fd) to head (406a64a).

Files Patch % Lines
cmd/access/node_builder/access_node_builder.go 0.00% 38 Missing :warning:
cmd/observer/node_builder/observer_builder.go 0.00% 37 Missing :warning:
...cutiondatasync/storage/pebble_datastore_manager.go 0.00% 23 Missing :warning:
storage/pebble/consumer_progress.go 0.00% 20 Missing :warning:
...executiondatasync/execution_data/execution_data.go 0.00% 16 Missing :warning:
...cutiondatasync/storage/badger_datastore_manager.go 0.00% 13 Missing :warning:
storage/pebble/operation/prefix.go 57.14% 12 Missing :warning:
storage/pebble/operation/jobs.go 0.00% 6 Missing :warning:
cmd/execution_builder.go 0.00% 5 Missing :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6180      +/-   ##
==========================================
- Coverage   41.49%   41.42%   -0.07%     
==========================================
  Files        2003     2008       +5     
  Lines      142757   142903     +146     
==========================================
- Hits        59232    59196      -36     
- Misses      77352    77532     +180     
- Partials     6173     6175       +2     
Flag Coverage Δ
unittests 41.42% <8.60%> (-0.07%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Jul 04 '24 17:07 codecov-commenter

I created a fork of ipfs/go-ds-pebble with your changes here: https://github.com/onflow/go-ds-pebble. please update the replace to this repo, then I'll approve

peterargue avatar Jul 31 '24 13:07 peterargue

I created a fork of ipfs/go-ds-pebble with your changes here: https://github.com/onflow/go-ds-pebble. please update the replace to this repo, then I'll approve

Thank you, @peter, for creating the forked repo. I have updated the replace.

UlyanaAndrukhiv avatar Jul 31 '24 14:07 UlyanaAndrukhiv