[Storage] [Feature request] Programmatically handle/bust Supabase's CDN cache for public URLs of replaced files with same name

[bennettdams]

Discussed in https://github.com/supabase/supabase/discussions/5737

Feature request

Is your feature request related to a problem? Please describe.

When you use a storage item's public URL, but the file with the same name got replaced (e.g. via supabase.storage .from("...").upload("..."), the CDN (Cloudflare) cache is still active.

Describe the solution you'd like

It would be nice if we as developers would be able to do bust the CDN cache programmatically via Supabase, because we know exactly when busting the cache would be necessary.

Problem description

I would like to e.g. save avatar images in the storage. In a perfect world, the image file name is the user ID, which makes it easy to handle the access rights for PUT/POST/DELETE requests.

In my app, a user can upload an image, which is uploaded to the storage via the Supabase client like so:

await supabaseServer.storage
    .from("...")
    .upload("user-id-123.jpg", avatarFileParsed, {
      upsert: true,
      contentType: 'image/jpg',
    })

This works fine. Now, I use the image's public URL for Next.js' image component, which also "caches" the image locally, but this is out of the scope of this question.

Looking at the response of a request for a storage image's public URL shows that it is cached via Cloudflare. It also has the cache control set for that (e.g. max-age).

This means that even though a new image has been uploaded to the Supabase storage, the CDN still uses the old image and therefore the image you receive for a public URL is also the old one - the app therefore shows the old image, even if a user clears his browser cache.

Describe alternatives you've considered

• Can one somehow use the cacheControl parameter of the upload function to bust the (CDN) cache?
• Can one maybe use URL params (at the end of a public URL) to trick the app to not use the cached version?
• Is there another way to bust the (CDN) cache?

My solution to circumvent this problem right now is to create a random ID for the filename on every upload and save that ID for each user in the database. That way, the CDN still works as intended, it's just that I don't have to bust the cache for a new/replaced image. This is obviously far from perfect, if not only for the additional code I had to write.

In a discussion for this topic, @rahul3v proposed an even better solution: Using something like updatedAt, which you typically already have for your user object, together with search params:

<image_url>?t=<update_time>

Additional context

Edit: It got mentioned by @GaryAustin1 that even Supabase's dashboard uses the second variant above (URL params) to show fresh data:

This works great for the use case of e.g. Supabase's dashboard, where you always want to see fresh data, but in a real-world app, where you don't know whether the cache should be fresh or not, I deliberately want to hit the cache if possible. It's not like you can bust the cache with the URL params "once" after a user uploads a new image - you'd need to always use the URL params and therefore will always miss the cache.

Proposed solution

We're already able to do upsert via .upload("..."), but this is only relevant for the file itself. Maybe the .upload("...") function could add another option parameter (something like bustCDNCache: true), which we as developers can use when replacing the same file.

[inian]

Hey @bennettdams,

There are a few ways to do this.

• Cache bust by adding the version parameter to your Supabase URL. By changing the version tag there, you can bust the CDN cache.
• You can set a shorter cacheControl header when you upload, so that the CDN checks for a new asset every max-age seconds. By default this is set to one hour, and you can set a shorter time there
• You can also bypass the cache by renaming the file itself instead of adding a version query parameter to the URL

We are looking to build a smarter cache here (eg. clearing the cache for the URL automatically when you use upsert), but that's not in the immediate roadmap.

[bennettdams]

@inian Thanks! Just to be sure: You didn't list a new option in your comment that I hadn't already mentioned in my post, right?

[inian]

yup, just confirming the alternatives you proposed Bennett

[413n]

That would be a really good feature to avoid workarounds to bust the cache on file update! +1

[fenos]

This is now been implemented by the Smart-CDN caching

[aboveyunhai]

@fenos @inian
Quick question to be confirmed, does it mean upload("same_url_as_before") now will automatically burst the cache? If someone else access the resource via same_url_as_before without any versioning param, and still benefit from the cache and get latest data if cache burst from our side. It's very common people wouldn't include a param when accessing a public resource, if people have to keep up with the version control/param to the lastest, then we completely give up the benefit of CDN. Assume we have the url with the latest versioning from the server, it will just create a double round trip issue (need to wait for the url version from the server and then call grab data via that url), this will be more slow.

[neos04]

This is now been implemented by the Smart-CDN caching

The smart CDN is restricted to only paid users so...🥲