Checkpoint note commitment trees and roots

[teor2345]

Motivation

Note commitment tree updates are very slow in Zebra right now, causing CI failures and instability, slow syncs, and poor user experience.

We can avoid most of these updates by checkpointing note commitment trees and roots.

Designs

Make zebra-checkpoints generate checkpoint files that store:

• all unique note commitment tree roots, indexed by their start height
• the note commitment tree frontiers for the mandatory and final checkpoints

In the finalised state:

• load the generated files
• store the checkpointed roots in the database
• when the configured checkpoint is reached, store the note commitment tree frontier in the database
• then update the trees and store the roots as in the existing state code

Write a new test to verify the root checkpoints. One way to do this is to query an existing cached state, which was generated without using the new checkpoints.

The PR should include an update to:

• sprout, sapling, and orchard note commitment trees
• state RFC database column design
• zebra-checkpoints documentation

Related Work

We might need to work out how to:

• convert from the zcashd RPC note commitment tree format to Zebra's format, or
• switch zebra-checkpoints to use Zebra RPCs, or
• add the missing zebra-checkpoints RPC to Zebra

But that can be a separate ticket.

If the checkpoint files are very large, we might also want to:

• download them, to avoid bloating the git repository (but this makes git diffs error-prone, and we might want to use a decentralised download method, or keep a fallback to full note commitment processing)
• store them as binary in the git repository (but this makes git diffs error-prone)
• store them as base64 in the git repository (but this means the hash encoding doesn't match the RPC hex encoding)