reth icon indicating copy to clipboard operation
reth copied to clipboard

Enable cache Optimization

Open malik672 opened this issue 11 months ago • 9 comments
trafficstars

Describe the feature

ref: https://github.com/paradigmxyz/reth/issues/12842#issuecomment-2497636181 ref: #12842 Image

The main reason this consumes so much storage or allocation I should say is because it's way too cache unoptimized, the whole fetcher.rs seems like it's not taking advantage of the cache instead there are more reads to the main memory instead, a common example should be

  1. Hash lookups that are not needed (this might seem efficient with 0(1) constant time indexing but it's not)
  2. Hash set with maximum size of 8 should be replaced with a Vec, of course this might not seem logical for indexing but for small data structure it's all about being contiguous in memory
  3. Scattered memory access, this leads to more cache miss

in all the reason for such allocation and latency is just because we are not using the cache efficiently,

@hai-rise @mattsse

Additional context

I don't think any amount of code removal or algorithmic reduction will make this more efficient apart from the using the cache efficiently

malik672 avatar Dec 20 '24 12:12 malik672