ouroboros-network icon indicating copy to clipboard operation
ouroboros-network copied to clipboard

Bring query utxo by address command performance on par with current version

Open dnadales opened this issue 3 years ago • 1 comments

QA reported that the query utxo --address command took about 10 seconds. The baseline (1.35.0 at the time of testing) took ~2.3 seconds. Furthermore, the query returned an empty set of addresses where 4 addresses were expected.

We need to fix the command and bring the performance of said command on par with the baseline.

This query is inefficient in LMDB because

  • we need to do disk access AND
  • we do it incrementally AND
  • we need to stow the ledger tables.

This query is inefficient in InMemory because

  • we do it incrementally AND
  • we need to stow the ledger tables.

Possible solutions:

  • InMemory can do the query all at once, no need to do it incrementally (8s > 5s).
  • Make the UTxO filtering functions target the CardanoLedgerTables and then stow the result (should take us to the theoretical limit).
  • Make this query faster, for instance by indexing by addresses.
  • Create a service to answer this query.

dnadales avatar Oct 11 '22 16:10 dnadales

@Jasagredo said:

For reference, returning the whole UTxO at once in the InMemory backend, measuring the time spent in stowLedgerTables (can ignore the bottom part, only the first 4 lines are relevant).

cardano-node timed

dnadales avatar Oct 11 '22 16:10 dnadales

We should deprecate this query in the node (not in the cardano-cli!), and move it to an external service that maintains this index. cardano-cli can provide this functionality through said indexing service. Efforts to implement this query in this way are tracked in this issue.

As for this issue, I think to close it we need to:

do the query all at once in the InMemory backend, no need to do it incrementally (8s > 5s).

CC: @Jasagredo @vfrsilva @disassembler

dnadales avatar Oct 18 '22 14:10 dnadales