Investigate Cacheing, Prefetching and Asynchronous Reads/Writes

[jon-chuang]

There are essentially 2 interrelated parts to this issue.

(1) Prefetching and asynchronous reads/writes is essentially the same thing. We simply want to use more threads to reduce load on the vm execution thread to reduce straight-line processing time of transactions, which bottlenecks the max theoretical TPS. A secondary objective is in line with the vision to enable chain performance scaling via parallelism.

The plan is to investigate the following approaches:

1. Only prefetch accounts that are available in the tx data
2. Optimistically execute future txs in parallel to keep cache warm (existing infra exists, but is single threaded 🤦‍♂️)
3. Introduce account state access metadata, read-only/write semantics enforced via VM calls (isn’t this necessary only for logic parallelisation though?). Unclear if should use existing cacheing infra (StateObjects, snaps). Investigate questions like: a. Tx failure upon undeclared access. b. Gas fees for access of declared accounts. c. Additional gas fees for undeclared state access (this ought to be very expensive and should reflect the total expected worst-case time to retrieve an undeclared account, against other gas table entries)

Since changes to disk can be delayed, it’s not clear that the current statedb already avoids laggard writes by committing changes to disk from memory via a separate thread. If it does something like writing in the same thread only at the end of the block processing and before block transmission or other actions, this would still be considered very stupid.

Things to investigate:

• Understand how the statedb stores the intermediate hash path, how and when it updates the root, cacheing structure, memory v.s. disk writes, detailed profiling analysis of bottlenecks via instrumented code.
• Understand how this issue relates to the flat kv initiative.
• How the VM communicates with the statedb and commits changes to data structures, especially those that can be quite large, like arrays, maps, etc. As long as StateObjects are updated in memory first and foremost, as long as execution is single-threaded, there will be no race condition. Nonetheless, one wonders if blocking locks should be introduced to guard against potential race conditions and for access control of modifications to state. Essentially, for execution without guarantee of committing, one must utilise copy-on-write.

(2) Cacheing. Although this is related to (1), it both has orthogonal rationale to exist, and also is intertwined with the intended full design. Thus, the current strategy of cacheing should be investigated carefully.