on-demand cache revalidation/incremental static genration

[anuragkumar19]

Describe the feature

Hi! The feature is to add a global function revalidate which can be called with path argument to revalidate the cache of a specific path. Ex: Initially route have {static:true} and in API route we can do revalidate('/blog'), revalidate('/blog/posts/1').

It will be useful for CMS and can be used with webhook to see instant changes.

This feature can further be extended in nuxt for on-demand incremental static generation.

This can be further simplified by providing a default route /__nuxt/revalidate/ or '__nitro/revalidate' with path='...' to revalidate the route. Of course, user can add middleware to secure the route to verify the request is coming from CMS.

Additional information

• [ ] Would you be willing to help implement this feature?

[pi0]

For incremental, we might use swr: true route rule it will presrve routes in filesystem/memory cache on demand and runtime. (static files cannot be removed from provider)

Assuming using cache API, You can create a server route (and protect it however you like) to clear cache using something like this:

export default eventHandler(async event => {
  // TODO: Protect route
  const storage = useStorage()
  const cacheKeys = await storage.getKeys('cache/nitro')
  await Promise.all(cacheKeys.map(key => storage.removeItem(key)))
  return { clearedItems: cacheKeys.length }
})

Can you try it and if found useful maybe we can make a built-in util for clearCache(prefix)

[ghost]

I proposed something similar here: https://github.com/unjs/nitro/discussions/716 I built two utils: createSwrCache and revalidateSwrCache stackblitz: https://stackblitz.com/edit/github-xc6cjm-qscwkg?file=utils/swrRevalidate.ts

[pi0]

Nice! What would be the usage of createSwrCache? (cache entries are internally and automatically created and managed)

We might expose two new utils, clearCache(prefix) (it is not essentially swr) and prefetchRoutes(routes: []) . Wdyt?

[ghost]

At that time I wanted to make a constistent way of configuring the cache-key for the storage-value with createSwrCache, which in turn made it clear what to target with revalidateSwrCache. But now I think it could be easier to have have a clearCache function on a cachedEventHandler itself.

 // routes/<route>.ts
export default cachedEventHandler(
  async () => {
    new Promise((resolve) => setTimeout(resolve, 1000));
    console.log('index');
    return `Response generated at ${new Date().toISOString()}`;
  },
  {
    swr: true,
    maxAge: 500000,
  }
);

And then import it elsewhere:

 // routes/<reval-route>.ts
import routeHandler from './<route>.ts'
export default defineEventHandler((event) => {
     routeHandler.clearCache()
});

The same might also be applicable to cachedFunction

Either that or a function that simply takes in a route-path.

[anuragkumar19]

@pi0 The above code you have given is invalidating all cache. It will be nice to have utils to invalidate specific cache.

Any reference to how cache key are generated?

[anuragkumar19]

I found it. https://github.com/unjs/nitro/blob/main/src/runtime/cache.ts#L177

Its ${first_16_character_of_url_path}.${hashed_path}

[anuragkumar19]

While going through issues and pull requests I found that It was planned but never implemented https://github.com/unjs/nitro/pull/545#issuecomment-1307446222

Edit: Can we ask danielroe he said that he had already a local prove-of-concept

[hcmlopes]

While going through issues and pull requests I found that It was planned but never implemented #545 (comment)

Edit: Can we ask danielroe he said that he had already a local prove-of-concept

I think the discussion mention which is happening on PR https://github.com/unjs/nitro/pull/545 is not quite what is being discussed here. I believe the discussions in that PR are specific to Vercel's cache and if so then I think this latest 3.7 release should have handled those requirements based on a PR of my own here https://github.com/unjs/nitro/pull/1723

I do think this specific on demand nitro cache invalidation still needs to be worked on. If Pooya provides some more detail I am willing to work on this, I am just a bit confused about this comment and what exactly prefetchRoutes(routes: []) would be doing or how it would work

[bernhardberger]

I've been looking at a solution for this myself recently. Imho it needs to be based on cache tags like in the https://github.com/dulnan/nuxt-multi-cache module.

That way one could leverage cache tags (e.g. Job Postings that are displayed and linked on several different pages/routes) and clear the associated cache tags from a CMS etc without having to know the exact routes the frontend is rendering that content (which is basically impossible to keep track of in larger projects where the editor is responsible for composing pages in the CMS)

[AdrianFahrbach]

Any updates on this?

[terion-name]

This is a really needed feature. Especially using ISR: true for dynamic pages (e.g. product page) – when underlying data changes (e.g. product was updated on backend) we need a way to revalidate it's page on CDN

[7iomka]

Any updates?

[Lea23VC]

Any updates?