exp/lighthorizon/services: Fetch ledgers in parallel to them being processed.
What
This modifies the ledger download+processing code to download subsets of the entire checkpoint range in parallel. Preliminary testing suggests that this can massively reduce request latency.
Before:
"GET http://localhost:8080/accounts/GBBJPGXTNNYBQGAIGONBFB6P7SUNO75GJEQ4Y7FVGMB2GPS3DJQVXSEP/transactions?cursor=179115963198013441&limit=10 HTTP/1.1" from 127.0.0.1:38834 - 200 199360B in 55.8414775s
After:
"GET http://localhost:8080/accounts/GBBJPGXTNNYBQGAIGONBFB6P7SUNO75GJEQ4Y7FVGMB2GPS3DJQVXSEP/transactions?cursor=179115963198013441&limit=10 HTTP/1.1" from 127.0.0.1:54188 - 200 199360B in 16.043121887s
(Note that the indices are local and no on-disk cache was used.)
Why
We can reduce latency by doing ledger downloads in parallel to processing. See #4468.
Known limitations
Obviously, neither of these represent acceptable levels of latency if we're talking about parity with classic Horizon. However, it's still a massive improvement over the previous code if we decide that high latency is acceptable for deeply historical requests.