image icon indicating copy to clipboard operation
image copied to clipboard

ipx cache generated images locally

Open jonatanlinden opened this issue 1 year ago • 37 comments

Hi,

I wonder if there is an option to cache images generated by ipx locally. Right now, it seems that ipx will regenerate a set of images every time npm run generate is run, which can get very cumbersome for 5000+ images. Previously, there seems to have been an option cacheDir as well as clearCache, but my impression is that these options have been removed (and maybe they didn't do what I imagine they did). Is there any plan to support such a feature? It might be interesting in cases of static site generation.

jonatanlinden avatar Mar 31 '23 15:03 jonatanlinden

This will be added as https://unstorage.unjs.io/drivers/fs , UnStorage is part of Nitro and this is engine of Nuxt 3. Information about adding this: https://github.com/nuxt/image/releases/tag/v1.0.0-rc.1

Triloworld avatar Jun 07 '23 06:06 Triloworld

I believe @pi0 is already working on this, with exciting things to come 😁

danielroe avatar Jun 08 '23 13:06 danielroe

Hi @danielroe, will this feature also be released in the v0 branch for Nuxt 2?

gmcinalli avatar Oct 20 '23 13:10 gmcinalli

It's unlikely unless there's a PR submitted to do it.

danielroe avatar Oct 20 '23 15:10 danielroe

The first step would be backporting IPX v2 for image v0 so PR welcome. (Ref: https://github.com/nuxt/image/pull/988 and https://github.com/nuxt/image/pull/1056)

pi0 avatar Oct 20 '23 15:10 pi0

Is a cache for nuxt generate available in the nuxt 3 version of nuxt image? I am only having ~100 images yet on a website, but nuxt generate already takes ~7 minutes.

fabianwohlfart avatar Nov 02 '23 22:11 fabianwohlfart

The generate part isn't even the biggest issue. When running in SSR/ISR we're seeing huge amounts of RAM usage on our server because IPX is constantly regenerating images.

Before ISR and static generate: we had like 50MB RAM usage. Now we have like 500MB-1GB.

Not what we envisioned when we went from a monolithic approach using a PHP CMS to moving headless using Nuxt. We don't have a CDN and also won't get one for doing image manipulation and storage so we'd very welcome a local image cache for IPX so it won't regenerate every time. IPX is great, but unusable in this form for production on SSR :-/

To get the scope of what's using those huge amounts of RAM: it's a one-pager with like 30 images in it total..

bernhardberger avatar Dec 23 '23 11:12 bernhardberger

@bernhardberger +1

CleanShot 2024-01-18 at 11 48 21@2x

This is what happens under heavy load, then node restarts... Expected behaviour would be local caching each time image is generated.

leoosa avatar Jan 18 '24 09:01 leoosa

Can anyone tell if a possible solution could be to cache the images via nginx?

Artem-Schander avatar Feb 22 '24 08:02 Artem-Schander

Can anyone tell if a possible solution could be to cache the images via nginx?

Not as long as the generated images don't have a unique hash.

bernhardberger avatar Feb 22 '24 11:02 bernhardberger

is there any work around for IPX catching on SSR ? my server is too humble to be able to handle this kind of regeneration on every request

w3bsite avatar Mar 05 '24 19:03 w3bsite

I would really appreciate local caching:

  • I have 20 images on one page, they come from AWS S3 and take 3-8s every time I reload the page (which happens a lot in development)
  • This also produces unnecessary traffic to my the file storage

MickL avatar Mar 05 '24 21:03 MickL

I was researching the same problem. In the release notes for v.1.0.0-rc.1 it mentions "Support multi-sources and server-side caching using Nuxt 3's built-in Storage Layer for the default image optimizer (unjs/ipx)"

But is this somethign different, or just not configured/implemented correctly?

It used to be on the roadmap for ipx v2: https://github.com/unjs/ipx/issues/171 but we are on v3 by now, and I think it is still not supported

https://github.com/unjs/ipx/issues/30

supermar1010 avatar Mar 19 '24 12:03 supermar1010

I found a workaround: Nuxt has a hybrid mode, with which you can prerender and cache routes. https://nuxt.com/docs/guide/concepts/rendering#hybrid-rendering

So what I have done is prerendering my index, which links all the images. This leads to nuxt also prerendering all the images. Thus not generating them on each request. The other option would be to cache the _ipx path with the swr option, this however did not work for me (nuxt returned a broken image from the cache), but I think this might be just a nuxt bug which is hoopefully fixed at some point.

supermar1010 avatar Mar 20 '24 10:03 supermar1010

The other option would be to cache the _ipx path with the swr option, this however did not work for me (nuxt returned a broken image from the cache), but I think this might be just a nuxt bug which is hoopefully fixed at some point.

This is the real issue and unfortunately a long standing bug. It makes SWR impossible to use as you only get broken images after the first cache miss.

Prerendering isn't a viable option usually when doing SSR as most people don't do SSR for the fun of it but because it's necessary in their (and my) use case. Prerendering is also problematic if you have lazy loaded components as those images won't be generated at all because the prerenderer can't reach them.

A working SWR however would migitate this issue massively.

bernhardberger avatar Mar 20 '24 14:03 bernhardberger

The other option would be to cache the _ipx path with the swr option, this however did not work for me (nuxt returned a broken image from the cache), but I think this might be just a nuxt bug which is hoopefully fixed at some point.

This is the real issue and unfortunately a long standing bug. It makes SWR impossible to use as you only get broken images after the first cache miss.

Prerendering isn't a viable option usually when doing SSR as most people don't do SSR for the fun of it but because it's necessary in their (and my) use case. Prerendering is also problematic if you have lazy loaded components as those images won't be generated at all because the prerenderer can't reach them.

A working SWR however would migitate this issue massively.

Actually, I found the solution to using SWR and IPX by setting '/_ipx/**': { swr: false, cache: false } in routeRules.

This doesn't cache ipx images, but at least it let's you use SWR for the rest of the site.

This really should get documented somewhere, it took me almost 1 year to find the solution to using ipx+swr and by the looks of it I'm not even remotely alone with this..

bernhardberger avatar Jul 18 '24 08:07 bernhardberger

Another solution would be to use Varnish Cache. This is a pretty drastic measure but it works well (for me at least)

Artem-Schander avatar Jul 18 '24 08:07 Artem-Schander

Another solution would be to use Varnish Cache. This is a pretty drastic measure but it works pretty well (for me at least)

Same here, added varnish cache in front of all /_ipx-requests, as quick fix while waiting for a better solution.

rhelling avatar Jul 18 '24 09:07 rhelling

Unfortunately Varnish isn't a solution, it's a mere workaround that isn't available to everyone..

bernhardberger avatar Jul 18 '24 12:07 bernhardberger