librustzcash icon indicating copy to clipboard operation
librustzcash copied to clipboard

`zcash_client_sqlite`: Add migration to link `utxo` table to `transactions`

Open str4d opened this issue 3 years ago • 2 comments

Currently the utxo table has columns for storing the transaction ID, and the mined block height. These are almost always duplicative of entries in the transaction table, and having them stored separately inhibits other possible relational queries.

We should implement a migration to:

  • Add a tx foreign key column.
  • Migrate existing utxo entries to reference matching transactions entries.
  • For those utxo entries that do not have matches, fetch the corresponding transactions from lightwalletd to populate the remaining transactions entries that the wallet assumes are populated during scanning.

The division of this work between the database migration and the network fetching might affect internals, but at a high level the migration API provided to zcash_client_sqlite users should hopefully be straightforward in the instructions it gives them.

str4d avatar Feb 02 '22 18:02 str4d

We discovered that is more challenging than we thought, because if we set the block height of the transactions entry, this requires an entry in the blocks table and we'd need at least the block hash to create such an entry; in addition, we now have the property "every entry in the blocks table corresponds to a block that we have scanned" so supporting this database refactoring will likely require other kinds of changes.

nuttycom avatar Sep 21 '23 15:09 nuttycom

This was partially closed by #1402. We added the migrations, but did not add logic to fetch extra data for the previously-missing transactions table rows, as it turned out the tables seem to work fine with just what we knew from the utxo table. We still need to fetch that data though, so leaving this issue open until we either do that, or move the remaining work into a new issue (which we might already have done without noticing, but it's a weekend so I'm not checking).

str4d avatar Jun 30 '24 02:06 str4d

This was completed by #1402, #1493, #1476

nuttycom avatar Oct 24 '24 21:10 nuttycom