hitbox
hitbox copied to clipboard
Instance based cache locks
Cache locks are the main protection from the dogpile effect. As main functionality (not related to some backend implementation) I suppose next simple structure.
Use something like a HasMap<&str, AsyncRwLock<CachedValue>>
where the key is a cache key of the current request and AsyncRWLock is an async version of classic RwLock (or some light pub-sub mechanic like tokio watch or one-shot channels).
How it should work:
- First request with cache key A to Upstream will create a record in a Hash table. and send a request to Upstream
- Second and next requests to Upstream with cache key A will check records on the hash table and subscribe to value changes (or asynchronous wait for changes)
- After first request will be resolved in Upstream cache should publish result to all consumers/waiters and remove record from HashMap