Feature Request: Stale While Revalidate

[KABBOUCHI]

something like this:

serve a stale cache between 5-10 seconds, but then update the cache in the background to keep it fresh

const metrics = await cache.getOrSet(
	'metrics',  
	
	async () => {
		return  [] // slow call here
	}, 
	
	{	 
	  ttl: '5 sec',
	  swr: '10 sec' 
	}
)

similar to https://x.com/laravelnews/status/1828542861890965752

[Julien-R44]

Yeah, actually today it's something that's already possible by using soft timeouts and grace periods:

bento.getOrSet({
    key: 'foo',
    factory: async () => return [],
    
    ttl: '4s',
    timeouts: { soft: '0ms' }, // Means, always returns stale value if we got one, and refreshes in the background
    gracePeriod: {
      enabled: true,
      duration: '10m', // Keep stale items for 10 minutes
    },
})

But indeed, the API is not very explicit/friendly. Maybe we should introduce a new method rather than a new option? Maybe getOrSetWhileRevalidate ? Still not sure

I'll think about it :)

[wodCZ]

Thank you for the example. Allow me a small correction:

With soft timeout set to '0ms' it throws that it is not a valid duration.

Error: Invalid duration expression "0ms"

Then I tried passing in 0 as an int, which seems to disable the soft timeout, acting as if there's no value in the grace period (i.e. it waits for the factory to complete).

Finally, with {soft: 1} it works correctly - result is returned immediately and value is refreshed in the background.

Here's the updated example:

bento.getOrSet({
    key: 'foo',
    factory: async () => return [],
    
    ttl: '4s',
    timeouts: { soft: 1 }, // Means, always returns stale value if we got one, and refreshes in the background
    gracePeriod: {
      enabled: true,
      duration: '10m', // Keep stale items for 10 minutes
    },
})

Also, from my understanding, stale while revalidate can be also achieved using earlyExpiration, for example:

bento.getOrSet({
    key: 'foo',
    factory: async () => return [],
    ttl: '10m',
    earlyExpiration: 0.01 // Means, refresh value in background if requested after 0.01 of ttl (6s)
})

[Julien-R44]

earlyExpiration is a bit different:

• We cache the foo entry for 10 minutes.
• At +15 minutes, a user requests the foo entry. It's not in the cache anymore, so no earlyExpiration is possible. The user will have to wait for the factory execution to complete.

So, in the end, it's a slightly different approach. In fact, you could indeed increase your ttl to keep your item longer but always refresh it early, and so have a sort of SWR.

Whereas if we use timeouts + grace period, it would look like this:

• We cache the foo entry for 10 minutes. The items are kept for an additional 10 minutes via GracePeriod. So in total, we keep everything for 20 minutes: 10 minutes fresh, and 10 minutes stale.
• At +15 minutes, a user requests the foo entry. It’s still in the cache but stale. So, we execute the factory, and if it exceeds 1ms of execution time, we return the stale entry while refreshing in the background.

The difference is subtle! But I wanted to clarify that😄

You are still right that we can't set a soft timeout of 0ms. So yeah, we need a more convenient API to handle SWR

[Julien-R44]

Planning to release a 1.0.0

In the next version SWR-like strategy will be super easy and enabled by default since this is what most users want and expect, I guess

bento.getOrSet({
    key: 'foo',
    ttl: '4s',
    grace: '2h',
    factory: async () => return [],
})

The only thing you need to do for it to work is to activate grace ( formerly gracePeriod.duration ). The softTimeout is set to 0 by default (and it won't throw like seen just above ).

I will write a few lines about that in the coming changelog for 1.0.0