[Nakamoto] address blocks by sighash in /v3/tenures/* endpoints used by the downloader

[jcnelson]

When downloading unconfirmed blocks, it will be necessary to address blocks by sighash instead of block ID because the signature set of the highest block is not guaranteed to be stable. Moreover, if there's a Stacks fork, it's possible that the same blocks can have multiple signature sets.

Quoting below (see https://github.com/stacks-network/stacks-core/issues/4810)

Writing down the conclusion of a huddle:

The endpoint GET /v3/tenures/{remote_tip_block_id}?stop={local_tip_block_id} works by requesting a range of unconfirmed tenure blocks ending (inclusively) with remote_tip_block_id (which is learned from a call to GET /v3/tenures/info), and starting (exclusively) with local_tip_block_id. This endpoint and associated chainstate need to be updated in three ways.

First, this endpoint must include the signer sighash of local_tip_block_id, so the remote node can (1) determine if the requester has the same view of the same fork even if the local_tip_block_id does not match any known block, and (2) determine if the requester has a different view of the signer signatures for local_tip_block_id so it can serve back at least the signatures for that block. The requesting node needs to handle the signature data for the block pointed to by local_tip_block_id by synthesizing a new block with these new signatures, but with the same data and header. Then, both nodes will have the same value for local_tip_block_id referring to the same block, as well as all blocks built atop it.

Second, remote_tip_block_id needs to be the block's signer sighash, not the block ID. This is because the signature data is not yet confirmed by signers, so it's possible that the block ID for this block can change in-between the call to GET /v3/tenures/info and GET /v3/tenures/{remote_tip_block_id}. By using the sighash value for remote_tip_block_id, instead of the block ID, we avoid having to deal with this race condition.

Third, the database schema for blocks should index block data by signer sighash in addition to block ID. Multiple block IDs can point to the same sighash, and each block ID would be associated with a unique set of signatures. Then, the block data and header (sans signatures) are stored at most once, but the node can synthesize a block from any signature set it has received. Multiple signature sets may be stored, but that's okay -- they would remain valid tips to build upon should signers require it.

[jcnelson]

Elaborating and simplifying this more -- it's sufficient to only require that remote_tip_block_id be the sighash OR block ID. This is because the block identified by remote_tip_block_id already commits to the block ID of its parent, as well as all of its ancestors. Therefore, local_tip_block_id can be a block ID. Furthermore, remote_tip_block_id would only need to be a sighash if the requested block is the chain tip; for any other block, remote_tip_block_id can be the block ID. So, the API endpoint should support both modes of operation (and because these are both sha512/256's, there's essentially zero chance that sighashes and block IDs can collide).

[jcnelson]

This will be closed by #4930

[blockstack-devops]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.