realtps
realtps copied to clipboard
Add optimizations for summing transactions across many blocks
After just a few days of running it takes over an hour to recalculate tps. This is likely to the enormous amount of networked file system I/O.
I don't have a full design in mind, but I'm thinking some kind of caching layer that stores the number of transactions across a range of blocks. Needs to be simple.
Perhaps something like
- While calculation is running it stores a vector cache of "runs" where each run is approximately 1 hour of blocks. Each run contains info like:
- first block number, hash, timestamp
- last block number, hash, timestamp
- next block number, hash, timestamp
- total number of transactions
- After calculation it stores the whole vector of runs to a file for the next calculation to use
- During calculation it traverses this cache before hitting the actual blocks, while building up a new cache
My 'blockruns' branch has work on this.