namada icon indicating copy to clipboard operation
namada copied to clipboard

Refactor shielded sync

Open batconjurer opened this issue 10 months ago • 3 comments

Describe your changes

This PR refactors shielded sync to make the following improvements

  • Allow fetching new masp txs and trial-decrypting notes to happen asynchronously
  • Parallelize the trial-decryptions
  • Modularize the logic so that we can mock parts of the algorithm for tests and to enable migration over to using a specila masp indexer
  • Added test coverage
  • Decouple nullifying notes and updating spent notes from the trial-decryption process
  • Refactor the masp.rs module in the sdk into several smaller files and submodules

The new architecture of the algorithm can be found here

N.B. The integration tests seem flaky on the CI, although I can't reproduce any of it locally. It's unclear to me if it is due to the changes in this PR.

Indicate on which release or other PRs this topic is based on

Checklist before merging to draft

  • [x] I have added a changelog
  • [x] Git history is in acceptable state

batconjurer avatar Apr 04 '24 15:04 batconjurer

Codecov Report

Attention: Patch coverage is 46.65658% with 1763 lines in your changes missing coverage. Please review.

Project coverage is 54.73%. Comparing base (883bd0f) to head (bdb61c3).

Files Patch % Lines
crates/sdk/src/masp/shielded_ctx.rs 51.29% 844 Missing :warning:
crates/sdk/src/masp/mod.rs 29.79% 436 Missing :warning:
crates/sdk/src/masp/utils.rs 51.28% 208 Missing :warning:
crates/apps_lib/src/client/masp.rs 0.00% 186 Missing :warning:
crates/sdk/src/masp/types.rs 72.61% 43 Missing :warning:
crates/node/src/bench_utils.rs 0.00% 27 Missing :warning:
crates/sdk/src/masp/test_utils.rs 89.79% 15 Missing :warning:
crates/apps_lib/src/cli/client.rs 0.00% 2 Missing :warning:
crates/node/src/lib.rs 0.00% 1 Missing :warning:
crates/sdk/src/tx.rs 0.00% 1 Missing :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3006      +/-   ##
==========================================
+ Coverage   54.05%   54.73%   +0.67%     
==========================================
  Files         315      319       +4     
  Lines      106296   107702    +1406     
==========================================
+ Hits        57461    58952    +1491     
+ Misses      48835    48750      -85     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Apr 10 '24 14:04 codecov[bot]

What's the status of this? Can we get it on a 0.35.1 base? @batconjurer @grarco

brentstone avatar May 09 '24 16:05 brentstone

I believe we haven't figured out yet the issue with the tests in CI

grarco avatar May 09 '24 16:05 grarco

Closed in favor of #3385

sug0 avatar Jul 09 '24 07:07 sug0