Documentation: link to Zebra architecture blog posts in zebrad overview for devs in the book

[dconnolly]

Motivation

We have a lot of detail in our book about design RFCs and other pages for zebrad, but lack a sort of high-level overview of the design and architecture, its services, how they communicate, etc. A consolidated overview would help new developers understand zebrad it faster.

We can link to some of our existing blog posts (https://www.zfnd.org/blog/a-new-network-stack-for-zcash/) in an overview document, especially as mempool is solidifying and client makes a small addition for the trial note decryption tasks, so the major architecture of zebrad is basically set.

Specifications

https://www.zfnd.org/blog/a-new-network-stack-for-zcash/ https://www.zfnd.org/blog/futures-batch-verification/ https://www.zfnd.org/blog/decoding-bitcoin-messages-with-tokio-codecs/

[dconnolly]

Hey team! Please add your planning poker estimate with ZenHub @conradoplg @jvff @mpguerra @oxarbitrage @upbqdn

[conradoplg]

Here's a graph (dot file) I generated to help me understand the dependencies between services (this is how it looks like after https://github.com/ZcashFoundation/zebra/pull/2741), and that we could add to the docs. If A points to B that means A is used by B (i.e. B has a reference to A). The dotted lines are circular dependencies which are sent back using channels.

digraph services {
state;
latest_chain_tip;
state -> chain_verifier;
state -> tx_verifier;
mempool;
state -> inbound;
chain_verifier -> inbound;
tx_verifier -> inbound;
mempool -> inbound [style=dotted];
peer_set -> inbound [style=dotted];
address_book -> inbound [style=dotted];
inbound -> peer_set;
latest_chain_tip -> peer_set;
inbound -> address_book;
latest_chain_tip -> address_book;
peer_set -> mempool;
peer_set -> syncer;
state -> syncer;
chain_verifier -> syncer;
peer_set -> sync_status;
state -> sync_status;
chain_verifier -> sync_status;
}

Render online

[dconnolly]

This

Here's a graph (dot file) I generated to help me understand the dependencies between services (this is how it looks like after #2741), and that we could add to the docs. If A points to B that means A is used by B (i.e. B has a reference to A). The dotted lines are circular dependencies which are sent back using channels.

digraph services {
state;
latest_chain_tip;
state -> chain_verifier;
state -> tx_verifier;
mempool;
state -> inbound;
chain_verifier -> inbound;
tx_verifier -> inbound;
mempool -> inbound [style=dotted];
peer_set -> inbound [style=dotted];
address_book -> inbound [style=dotted];
inbound -> peer_set;
latest_chain_tip -> peer_set;
inbound -> address_book;
latest_chain_tip -> address_book;
peer_set -> mempool;
peer_set -> syncer;
state -> syncer;
chain_verifier -> syncer;
peer_set -> sync_status;
state -> sync_status;
chain_verifier -> sync_status;
}

Render online

This is awesome! I wonder if we can do a service dependency diagram at a similar high level automatically derived from the codebase, so that we can keep it up to date automatically and embed the result in our zebra.zfnd.org docs / rust docs

[conradoplg]

This is awesome! I wonder if we can do a service dependency diagram at a similar high level automatically derived from the codebase, so that we can keep it up to date automatically and embed the result in our zebra.zfnd.org docs / rust docs

I think it's doable with https://crates.io/crates/syn, parsing start.rs, but I'm not sure if it's worth the effort :thinking: Would be cool though :laughing:

[mpguerra]

Hey team! Please add your planning poker estimate with ZenHub @conradoplg @jvff @mpguerra @oxarbitrage @upbqdn

We should remove anyone who hasn't yet voted and set the final estimate here, I can't do it because I didn't start the vote... :) @dconnolly can you do the honours?

[teor2345]

I'm wondering if we could just link to the blog posts from the book instead?

[teor2345]

I'd like to do this task after we've made some more decisions about client support, because they will impact dependencies and architecture.

[mpguerra]

I'm wondering if we could just link to the blog posts from the book instead?

if we do this we should make sure the existing blog posts are going to be migrated to the new website