Reth Book - Stages framework/draft

[0xKitsune]

I started a draft for the stages chapter in the book but wanted to stop before going too far to get some feedback on the direction. I would love to hear any thoughts on the overall tone/direction, as well as if you think it should be more code heavy/technical (I tried to keep it very high level).

Im happy to adjust towards whatever direction might be best for the book. Let me know what you think.

[codecov-commenter]

Codecov Report

Merging #360 (7221b28) into main (d66138e) will increase coverage by 1.04%. The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #360      +/-   ##
==========================================
+ Coverage   73.71%   74.75%   +1.04%     
==========================================
  Files         230      237       +7     
  Lines       21895    22641     +746     
==========================================
+ Hits        16139    16925     +786     
+ Misses       5756     5716      -40     

Impacted Files	Coverage Δ
crates/net/eth-wire/src/types/state.rs	94.59% <0.00%> (-5.41%)	:arrow_down:
crates/net/eth-wire/src/types/receipts.rs	96.22% <0.00%> (-3.78%)	:arrow_down:
crates/net/eth-wire/src/types/broadcast.rs	62.29% <0.00%> (-2.71%)	:arrow_down:
crates/net/eth-wire/src/p2pstream.rs	82.62% <0.00%> (-2.25%)	:arrow_down:
crates/stages/src/db.rs	71.96% <0.00%> (-1.68%)	:arrow_down:
crates/transaction-pool/src/validate.rs	70.83% <0.00%> (-1.51%)	:arrow_down:
crates/net/eth-wire/src/types/blocks.rs	94.78% <0.00%> (-1.29%)	:arrow_down:
crates/net/network/src/message.rs	38.94% <0.00%> (-1.28%)	:arrow_down:
crates/net/eth-wire/src/types/transactions.rs	97.80% <0.00%> (-0.84%)	:arrow_down:
crates/net/network/src/config.rs	62.83% <0.00%> (-0.81%)	:arrow_down:
... and 151 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[0xKitsune]

@rkrasiuk Thanks for all the comments, Ill get those fixed up later today.

[akirillo]

this is for later, but when we host the code docs on docs.rs, we should include links to the docs.rs pages for the stages crate, pipeline.rs, and each of the stages we mention

[akirillo]

I like this - focusing on a human-readable "what this does" + "how these things fit together" as opposed to a granular "how this works" at the level of individual functions which should be handled by code docs. I don't think you need to get more technical than this, it's great

As you continue on this, one thing that should definitely be included before we push is some rationale explaining why we are using the staged sync architecture, i.e. why this design is a good one. Currently the best written rationale on this that I've found is in this Erigon blog article, but it would be great to get more seasoned input from the rest of team.

Maybe we could also touch on the developer story of e.g. adding a stage to the pipeline? Just trying to focus on the book being how users interact with the components, but maybe this is obvious? Not sure, would love to hear your thoughts as well

[akirillo]

FWIW, when you have the time, try running a cargo doc --open for the repo and seeing what sorts of code docs have been written, to check that there's a reasonable boundary between them and the book. E.g., take a look at the docs generated for the Pipeline struct (target/doc/reth_stages/struct.Pipeline.html), they also get into some high-level explanation with a nice graph and I would love your thoughts on where that sort of content should live

Both can be subject to change, don't consider anything set in stone!

[0xKitsune]

@rkrasiuk Thanks for all the comments, I updated the draft with all of those fixes.

@akirillo

As you continue on this, one thing that should definitely be included before we push is some rationale explaining why we are using the staged sync architecture, i.e. why this design is a good one. Currently the best written rationale on this that I've found is in this Erigon blog article, but it would be great to get more seasoned input from the rest of team.

Sounds good, I will keep this in mind and also check out the link. Feel free to send anything else over that you think may be helpful.

FWIW, when you have the time, try running a cargo doc --open for the repo and seeing what sorts of code docs have been written, to check that there's a reasonable boundary between them and the book.

Great, I will check this out later tonight as well.

[0xKitsune]

love the direction and in favor of quick-merging these PRs and doing fast-followons instead of having them live for too long.

Let's also add a note in the top-level README that the book is continuously rendered at https://paradigmxyz.github.io/reth/?

Sounds good to me, I am working on updating the draft and pushing to get the chapter all wrapped up before the end of the weekend.