Conseil
Conseil copied to clipboard
The indexer is terribly slow due to fees processing
As shown by the query engine the way fees are collected takes a long time. The process is run once every N cycles of Lorre and loads past fees.
Analysis:
As the number of fees grows the indexing to collect distinct
values becomes useless and the engine is forced to do a linear scan.
The query is: select distinct "fee", "timestamp", "cycle", "level" from "tezos"."operations" where "kind" = 'transaction' order by "timestamp" desc limit 1000
the longest amount of time is being taken up by the sort and the subsequent sequential scan, please keep in mind that kind is already an index plan is limit(.086ms) -> Unique(2.406 ms) -> sort(12,105ms) -> seq_scan(11,221ms)
Proposed solutions:
Put a lower bound on the query timestamp to reduce the number of rows returned for the computation when doing the distinct
operation.
Forego the distinct
requirement, if it's not necessary (why should it happen that we collect more than one unique fee?)