stellar-core
stellar-core copied to clipboard
Prefetch entries in Soroban ops
InvokeHostFunctionOp, BumpFootprintExpirationOp, and RestoreFootprintOp does not prefetch any entries, leading to each entry in the footprint being fetched individually.
Setting target milestone to Post testnet
after discussing with @sisuresh
It looks like Soroban TXs (Restore, Extend, InvokeHostFunction) don't prefetch any keys. For SQL, this makes sense. We prefetch all keys in a single query, so there is no way for us to enforce byte read limits. We could potentially do something like this but the query performance may be significantly limited by such measures, defeating prefetch gains in the first place. However, for BucketListDB, we can enforce a byte limited prefetch, where we prefetch keys until we reach the given limit. This would provide more efficient disk access but prevent IO DOS attacks, where a malicious TX includes lots of very large keys with a small readBytes value. This will require some plumbing to make sure we propagate the prefetch byte limit correctly since this would have to be a per-tx limit. For SQL, I think it's best that we just permanently disable Soroban TX prefetching since we're shifting to BucketListDB anyway and a SQL solution doesn't seem super clear.
Agree. I think for BucketsDB the benefit is batching should allow us to take advantage of the order of keys and reduce random seeks (and may also yield benefits down the line with parallelization as well). Assigning to @ThomasBrady that will probably be the one working on this at some point