De-duplicate note commitment trees in the finalized and non-finalized state

[teor2345]

Motivation

Zebra stores duplicates of each Sprout note commitment tree, and after #3563, it also stores duplicates of each Sapling and Orchard note commitment tree.

This is a RAM, disk space, and speed optimisation.

Major Optimisations

• [ ] #4766

• [ ] #4721

• [ ] #4784

• Only store the first non-finalized sprout/sapling/orchard anchor in each identical series of anchors, and access them as in #4784. This needs a full sync performance test.

• Use TransactionIndex for the value in Chain.tx_by_hash (after #3152 merges)

Minor Cleanups

These cleanups are optional.

1. Remove the non-finalized Chain.{sapling, orchard}_note_commitment_tree variables, and look them up in Chain.{sapling, orchard}_trees_by_height using the maximum chain height.

2. Change the non-finalized Chain.sprout_note_commitment_tree to sprout::Root. Look up the actual note commitment tree in Chain.sprout_trees_by_anchor.

3. Change the finalized sprout_note_commitment_tree to a key of () and a value of sprout::Root. Look up the actual note commitment tree in sprout_anchors.

Related Work

Follow-Up From:

• #3563

[ftm1000]

Hey team! Please add your planning poker estimate with ZenHub @conradoplg @dconnolly @jvff @oxarbitrage @teor2345 @upbqdn

[teor2345]

We confirmed that this ticket is optional in https://github.com/ZcashFoundation/zebra/pull/3818#discussion_r823141591

[teor2345]

We kept this open because note commitment trees are the most CPU-intensive work that we do.

[teor2345]

I think we've finished the urgent parts of this ticket now.