[EPIC] Include block hash in index keys

[Ullaakut]

Description

In order to allow the live indexer to index live data before it is sealed, the DPS needs to change its indexing process to allow multiple versions of an index at a given height, in the case of a fork. In order to do this, we can change our indexes to include the block hash.

This will reduce the latency between the time at which events happen and they are available in our API, which is essential for the DPS to be relevant in the Flow ecosystem.

The consensus tracker currently already accesses block data as soon as they are finalized, while the execution tracker has access to all execution data that is uploaded onto its GCP bucket. All finalized blocks get pushed onto the execution tracker's queue, to be downloaded in the same order as they were finalized.

To double check: The consensus tracker follows the access nodes to bind its OnBlockFinalized so I assume that should be the case and we don't need to change anything.

To double check: The block order is very important in the execution tracker, since we need to read trie updates sequentially. Since trie updates contain the previous state commitment they apply to, as long as for each "fork" we get updates in the right order, the current implementation should work.

TODO: Once this is all confirmed, create sub tasks and start implementation.

Changes

• Change the consensus follower so that it can warn us about executed blocks
• Instead of downloading finalized blocks only, we need to download block data as soon as they get optimistically executed.
• Instead of pruning forests when we reach a matching commit with the expected block hash, we need to prune them when a block gets finalized.
  • The same goes for the execution tracker
• Update the index to use block hash in keys
  • Update the DB Schema documentation in Flow DPS to match with the new tables
  • Update the DPS API to use the new index keys & update its documentation
  • Update the Flow DPS Rosetta API to use the new index keys
  • Update the Flow DPS Access API to use the new index keys
• Since we download block data for blocks that haven't been sealed yet, we need to add a sanity check to ensure that our index matches with the sealed execution state as we go.

Additional Notes

• This is a major breaking change, and will render previously built indexes obsolete. -> DPS v2.0.0 already coming up 😄
  • Same goes for the Rosetta and Access API repos, they need a new release that makes it clear the change is breaking.
• This means the DPS API also needs to be updated to use block hashes for its queries.
• This will impact the DPS Access API and DPS Rosetta API repositories, which will no longer be compatible with the previous version of the DPS.