stretto
stretto copied to clipboard
Get-or-insert-with semantics / Dogpile Effect Mitigation
Async Caches benefit from being able to ensure that - if a value is not currently cached - a future imminently fulfilling the entry will be passed to the other asynchronous readers. I haven't found a way to implement this pattern without adding another layer of internal mutability within the existing Stretto cache; is there an officially endorsed way to deduplicate cache fulfillment requests? If not, I'd like to request the feature.
Hi Dessix, thank you suggestion. Currently, stretto does not provide a function get_or_insert
like Hashmap
or etc. I will add something like this in next release. Besides, would you mind to provide a mock example for your use cases for this feature request?
The important part is that it only calls the factory function to create the value when the value is missing- but it also prevents any subsequent calls from beginning construction while it is in the process of fulfilling the first.
A more concrete example is if I have an LRU cache to keys on a remote key/value store, and want to ensure that I only query the remote store once if my cache is missing the value- even if I suddenly receive many requests for it at once. Once it's fulfilled, all active requesting tasks can share the result.
c.get_or_try_fulfill_with(|k: &Key| async move {
// some expensive, async operation to retrieve a value
let response = myExpensiveRequest.await?;
Ok(response)
})