CacheTower icon indicating copy to clipboard operation
CacheTower copied to clipboard

Concept - Unpopular cache items don't forward propagate

Open Turnerj opened this issue 4 years ago • 1 comments

https://blog.cloudflare.com/why-we-started-putting-unpopular-assets-in-memory/

For Cloudflare, they wanted to prevent unnecessary writes to SSDs to keep them around longer so they didn't want to have unpopular cache items in their SSD cache. On top of the lifetime benefits, it also increased overall performance as writes-while-reading slows down individual chips on an SSD.

While CacheTower is unaware of the storage medium, it would be interesting to implement a similar approach. If we track how often an item in the cache is hit, we can track whether we should even back populate it from in-memory to whatever other cache layers are configured.

This can prevent unnecessary writing and storage for "one-hit-wonders" in deeper layers of the cache stack. If you are using Redis, it would take pressure off the Redis server. If you are using Disk, it would take pressure off the disk.

At minimum we would need:

  • To track every cache access (As a uint perhaps?)
  • Update back populate logic to check if cache access is often or not

Considerations:

  • Does this only work for in-memory cache?
  • If not, does this work just for the first cache layer?

Turnerj avatar May 12 '20 01:05 Turnerj

As an aside to this, potentially establishing an Least Recently Used (LRU) eviction strategy. While I can't measure the memory used in the cache (I don't serialize it, I just reference it), maybe there is some backend API I can use to gauge memory usage? Or just do it dumbly by the number of items?

Turnerj avatar May 14 '20 13:05 Turnerj