witnet-rust
witnet-rust copied to clipboard
witnet
Fix performance of getBalance and getUtxoInfo methods
The getBalance method is implemented by iterating over all the existing Unspent Transaction Outputs (UTXOs). This was good enough when the UTXO set was stored in memory, but with the recent changes from #2159 the UTXO set is now not in memory, so iterating the UTXO set is slow. The same happens with the getUtxoInfo method, which needs to iterate the UTXO set to find UTXOs that belong to the provided address.
This is a table with the approximate time it takes to execute each method, in milliseconds:
| Method \ Witnet version | 1.5.0 | 1.5.1 |
|---|---|---|
| getBalance | 4300 | 4100 |
| getBalance (simple) | 80 | 3950 |
| getBalance (simple, own address) | 80 | 15 |
| getUtxoInfo | 100 | 4000 |
| getUtxoInfo (own address) | 15 | 15 |
The reason why querying info about the node's "own address", is because there is a data structure called OwnUnspentOutputsPool which serves as a cache of the UTXOs that belong to this node. This data structure is needed to implement the commit collateral creation, as well as the methods to create value transfer and data request transactions.
So one way of fixing the performance of this methods would be to implement something similar to the OwnUnspentOutputsPool, and have a map of "address" to "list of UTXOs". This would make all the methods take around 15ms but at the cost of consuming a few GB of memory, and making every block consolidation slightly slower. So if implemented that way it should be an optional feature.
So one way of fixing the performance of this methods would be to implement something similar to the OwnUnspentOutputsPool, and have a map of "address" to "list of UTXOs". This would make all the methods take around 15ms but at the cost of consuming a few GB of memory, and making every block consolidation slightly slower. So if implemented that way it should be an optional feature.
Yes, I was watching this issue. I would totally be supportive of an implementation like this where I can start a node with a --utxos-in-memory flag or something similar. I think the extra RAM usage (which we are seeing now anyway) is worth the trade-off versus the explorer-fetched balance becoming virtually unusable.
