Support for revalidation using loaders that set expiry

[tschaub]

I'm unclear if this is currently supported, but here is my use case:

1. I would like to cache values that are expensive to look up (fetching the value remotely)
2. After looking up a value, I know the expiry information (e.g. an exp claim in a JWT)
3. I would like to asynchronously revalidate the value before the expiry is reached (e.g. get the latest value of a rotated JWT)

I see that I can configure a hot.NewHotCache with a loader using WithLoaders. I think that takes care of 1 above.

I see that I can call cache.SetWithTTL to set the TTL for a value in the cache. In my case, I know the appropriate TTL value in the loader itself (e.g. after fetching a JWT and parsing the exp claim). It doesn't seem like it would work to call cache.SetWithTTL within the loader, but maybe I could defer this call until after the loader returns? In any case, I'm not sure how to deal with 2 above.

The docs make it sound like revalidation happens after the TTL. So data becomes stale after the TTL. Is it possible to separately configure the stale and expiry times? In my case, I would like to consider data stale before it expires (so the latest remote data can be fetched asynchronously while serving stale data). I've used github.com/karrick/goswarm in the past, and with that the loader is called asynchronously after the GoodStaleDuration and called synchronously after the GoodExpiryDuration - so some people may get stale data (not the latest of the rotated JWT), but nobody will get expired data (an expired JWT). The limitation of the goswarm package is that I have to configure the expiry at the cache level, and in my case I don't know the expiry until I load the data. So I was curious if this package might be able to address 3 above.

[samber]

What about a new Loader method that returns both the value and TTL ?

Does returning ttl - staleDuration look good to you or do you need a custom mode telling hot to embed the stale duration in ttl duration ?

[tschaub]

What about a new Loader method that returns both the value and TTL ?

I think something like this would be very nice:

--- a/loader.go
+++ b/loader.go
@@ -1,5 +1,18 @@
 package hot
 
+import "time"
+
+type ValueDetail[V any] struct {
+       Value        V
+       StaleAfter   time.Duration
+       ExpiresAfter time.Duration
+}
+
+// LoaderWithDetail is a function type that loads values with additional details for the given keys.
+// It should return a map of found key-value pairs and an error if the operation fails.
+// Keys that cannot be found should not be included in the returned map.
+type LoaderWithDetail[K comparable, V any] func(keys []K) (found map[K]*ValueDetail[V], err error)
+

After a value is cached, if it is accessed before the StaleAfter duration elapses, then the cached value is returned. If the StaleAfter duration has elapsed, but the ExpiresAfter duration has not yet elapsed, then the (now stale) value is returned and the loader is called asynchronously to revalidate the value. If the ExpiresAfter duration has elapsed, then the cached value is not returned and the loader is called synchronously to get a new value. This way nobody gets expired data, and the only time a caller has to wait for data to be loaded is if the ExpiresAfter duration has elapsed before any other caller has tried to access the data (and the loader has returned a fresh value).

Maybe too subtle, but the StaleAfter and ExpiresAfter details above could also apply to cache behavior when the loader returns an error.

I'm not sure the same behavior (never returning an expired value, revalidating asynchronously) can be achieved with only a single TTL setting (unless there is hardcoded logic like revalidating if half of the TTL has elapsed or something).