nano-node
nano-node copied to clipboard
Add reverse link store for fast receive block lookups by only their send block hashes
- Added reverse_link_store to persist the [send block hash] -> [receive block hash] pairs.
- Implemented LMDB v21 -> v22 migration to create the new reverse_links table on startup.
- Implemented node.enable_reverse_links option (default disabled)
- Hooked into block_cemented_callback to store the reverse links of new cemented blocks.
- I'm still not sure if this is the best place to do that, but it seems to work.
- Implemented initial reverse link pair loading into the node constructor.
- I'm still not sure if this is the best place to start this relatively long loading process (it took around an hour on my system to process all cemented blocks from the live ledger), but it works.
- Added some reverse link test cases
- Test case for the initial reverse link loading in the nano::node constructor is missing, because I was not yet able to find a proper solution to modify/inject the nano::store before nano::node creation.
This PR will drastically improve the performance of #3702
@dsiganos @thsfs Commit squash & rebase done (PR includes the iterable_store implementation from #3766).
@dsiganos , @clemahieu , how the reverse links table could get affected by the confirmation height?
I think the reverse_link table should be outside of the core database and in a separate auxiliary database. That will:
- avoid the core database versioning problem
- allow flexibility
- separation of core data and auxiliary data
I think the reverse_link table should be outside of the core database and in a separate auxiliary database. That will:
- avoid the core database versioning problem
- allow flexibility
- separation of core data and auxiliary data
Oof, this would make the persistence layer even more complicated than it already is. In addition these changes would have to be made for both lmdb and rocksdb. I'm not really convinced it's worth the effort, especially if there are plans to move to an SQL backend.
I will update the PR within the next two days. Have to finish another task first, but should be done soon.
I will update the PR within the next two days. Have to finish another task first, but should be done soon.
Done. I also readded the LMDB reverse_links table creation and increased the store version.