hitbox icon indicating copy to clipboard operation
hitbox copied to clipboard

Instance based cache locks

Open singulared opened this issue 3 years ago • 0 comments

Cache locks are the main protection from the dogpile effect. As main functionality (not related to some backend implementation) I suppose next simple structure.

Use something like a HasMap<&str, AsyncRwLock<CachedValue>> where the key is a cache key of the current request and AsyncRWLock is an async version of classic RwLock (or some light pub-sub mechanic like tokio watch or one-shot channels).

How it should work:

  1. First request with cache key A to Upstream will create a record in a Hash table. and send a request to Upstream
  2. Second and next requests to Upstream with cache key A will check records on the hash table and subscribe to value changes (or asynchronous wait for changes)
  3. After first request will be resolved in Upstream cache should publish result to all consumers/waiters and remove record from HashMap

singulared avatar Dec 04 '21 17:12 singulared