CacheTower
CacheTower copied to clipboard
Concept - Unpopular cache items don't forward propagate
https://blog.cloudflare.com/why-we-started-putting-unpopular-assets-in-memory/
For Cloudflare, they wanted to prevent unnecessary writes to SSDs to keep them around longer so they didn't want to have unpopular cache items in their SSD cache. On top of the lifetime benefits, it also increased overall performance as writes-while-reading slows down individual chips on an SSD.
While CacheTower is unaware of the storage medium, it would be interesting to implement a similar approach. If we track how often an item in the cache is hit, we can track whether we should even back populate it from in-memory to whatever other cache layers are configured.
This can prevent unnecessary writing and storage for "one-hit-wonders" in deeper layers of the cache stack. If you are using Redis, it would take pressure off the Redis server. If you are using Disk, it would take pressure off the disk.
At minimum we would need:
- To track every cache access (As a
uint
perhaps?) - Update back populate logic to check if cache access is often or not
Considerations:
- Does this only work for in-memory cache?
- If not, does this work just for the first cache layer?
As an aside to this, potentially establishing an Least Recently Used (LRU) eviction strategy. While I can't measure the memory used in the cache (I don't serialize it, I just reference it), maybe there is some backend API I can use to gauge memory usage? Or just do it dumbly by the number of items?