Add an architecture diagram to illustrate the moving parts

[sabbyanandan]

As additional guidance to developers, it would be helpful to add an architecture diagram to illustrate the moving parts so developers can relate to what state stores (in-memory, Redis, RDBMS, etc.) and prepare for the deployment topology in their environment.

Additionally, it will help unpack the value proposition of Chainhook in a blog/docs, including how the speedier indexing is accomplished or faster data retrieval (eg: Redis) can reduce the latency and the likes.

[MicaiahReid]

@ryanwaits, I was working on some diagrams as part of a Chainhook deep dive. Idk if these are helpful since it's mostly internals of Chainhook, but I'll put them here in case: Chainhook service high level architecture (with the sub components below):

flowchart TD
  start["`chainhook service start`"] --> run
  redis --Load Predicates -->run
  run --> tsv_check["`check if config specifies tsv`"]
  tsv_check -- Yes --> tsv["`consolidate_local_stacks_chainstate_using_csv`"]
  tsv_check -- No --> next["Spawn Threads"]
  tsv --> next
  next --> stacks_scan
  next --> btc_scan
  next --> predicate_api
  next --> event_observer
  next --> observer_event_lp
  subgraph stacks_scan["start_stacks_scan_runloop thread"]
  end
  subgraph btc_scan["start_bitcoin_scan_runloop thread"]
  end
  subgraph event_observer["start_event_observer thread"]
  end
  
  subgraph observer_event_lp["Listen for Observer Events"]
  end
  subgraph predicate_api["start_predicate_api_server thread"]
  end
  stacks_scan --> user
  btc_scan --> user
  predicate_api["start_predicate_api_server thread"] <--> user
  event_observer --> user
  observer_event_lp <--> redis[(Predicates Redis DB)]
  observer_event_lp --> stacksdb[(Stacks Block DB)]
  stacksdb --> stacks_scan

start_stacks_scan_runloop Thread (also applies to the start_bitcoin_scan_runloop thread in the above):

flowchart TD
    stacks_scan_op_rx --> new_p["Check for new predicate to scan"]
    new_p --Yes -->scan_stacks_chainstate_via_rocksdb_using_predicate
    new_p --No --> stacks_scan_op_rx
    subgraph scan_stacks_chainstate_via_rocksdb_using_predicate
        get_block_heights_to_scan --> lp1["More blocks to scan?"]
        lp1 --Yes --> evaluate_stacks_chainhook_on_blocks
        lp1 --No --> terminate_thread
        evaluate_stacks_chainhook_on_blocks --> matches["Predicate matches block?"]
        matches -- Yes --> tr["Trigger Predicate Action"]
        matches -- No --> err["Predicate Scan/Trigger Failed?"]
        tr --> err["Predicate Scan/Trigger Failed?"]
        err -- Yes --> set_predicate_interrupted_status
        err -- No --> ten["Ten Blocks Scanned?"]
        set_predicate_interrupted_status --> terminate_thread
        ten -- Yes --> set_predicate_scanning_status
        ten -- No --> unconfirmed
        set_predicate_scanning_status --> unconfirmed
        unconfirmed["Block height > predicate end block?"]
        unconfirmed -- Yes --> block_conf["Is block confirmed?"]
        unconfirmed -- No --> lp1
        block_conf -- Yes --> set_confirmed_expiration_status
        block_conf -- No --> set_unconfirmed_expiration_status
        set_confirmed_expiration_status --> lp1
        set_unconfirmed_expiration_status --> lp1
    end

Listen for Observer Events Thread:

flowchart TD
  subgraph observer_event_lp["Listen for Observer Events"]
    new["New Event"] --> StacksChainEvent
    new["New Event"] --> PredicateEnabled
    new["New Event"] --> PredicateRegistered
    new["New Event"] --> PredicateDeregistered
    new["New Event"] --> BitcoinChainEvent
    new["New Event"] --> PredicateInterrupted
    PredicateRegistered --INSERT --> redis["Predicates redis DB"]
    PredicateDeregistered --DEL --> redis["Predicates redis DB"]
    PredicateEnabled --PUT --> redis
    BitcoinChainEvent --PUT --> redis
    StacksChainEvent --PUT/DEl --> stacksdb[("Stacks rocksdb")]
    StacksChainEvent --> redis
    PredicateInterrupted --PUT --> redis[("Predicates redis DB")]
  end

[MicaiahReid]

Here's a more end-user focused diagram. These are the states that a predicate can go through:

stateDiagram
    state to_stream1 <<choice>>
    state to_stream2 <<choice>>
    state to_exp1 <<choice>>
    state to_exp2 <<choice>>
    [*] --> New
    New --> to_stream1
    to_stream1 -->Scanning:shouldScan 

    to_stream1 -->Streaming: shouldStream
    Scanning --> to_stream2
    to_stream2 -->Scanning: shouldKeepScan
    to_stream2 -->Streaming: shouldStream
    to_stream2 -->UnconfirmedExpiration: shouldUnconfirmedExpire
    to_stream2 -->ConfirmedExpiration: shouldExpire
    Streaming --> NewBlockMined
    NewBlockMined -->to_exp1
    to_exp1 --> Streaming: shouldStream
    to_exp1 --> UnconfirmedExpiration: shouldUnconfirmedExpire
    to_exp1 --> ConfirmedExpiration: shouldExpire
    UnconfirmedExpiration --> NewBlockConfirmed
    NewBlockConfirmed --> to_exp2
    to_exp2 --> UnconfirmedExpiration: shouldUnconfirmedExpire
    to_exp2 --> ConfirmedExpiration: shouldExpire
    ConfirmedExpiration --> [*]
    %% note right of to_stream1
    %%     shouldScan = start_block <= chain_tip
    %% end note
    %% note left of to_stream1
    %%     shouldStream = current_scan_block > chain_tip && current_scan_block < end_block
    %% end note
    %% note left of Scanning
    %%     shouldKeepScan = current_scan_block <= chain_tip && current_scan_block < end_block
    %% end note
    %% note left of to_exp1
    %%     shouldUnconfirmedExpire = current_scan_block == end_block && !isConfirmedBlock(current_scan_block)
    %% end note
    %% note right of to_exp2
    %%     shouldExpire = current_scan_block == end_block && isConfirmedBlock(current_scan_block)
    %% end note

Key: shouldScan = start_block <= chain_tip shouldStream = current_scan_block > chain_tip && current_scan_block < end_block shouldKeepScan = current_scan_block <= chain_tip && current_scan_block < end_block shouldUnconfirmedExpire = current_scan_block == end_block && !isConfirmedBlock(current_scan_block) shouldExpire = current_scan_block == end_block && isConfirmedBlock(current_scan_block)