Bug report

Describe the bug

The public url from storage responds with etag and last modified headers. The problem is that subsequent calls will never get 304 response, as if it ignores the etag and the last modified headers.

To Reproduce

1st call

# request
curl -v --request GET \
  --url https://rntpzayauzmhsmxjjtfz.supabase.co/storage/v1/object/public/images/test-box-image.jpeg \
  1> /dev/null

# response
< HTTP/2 200 
< cache-control: max-age=3600
< etag: "76d19e08653c1055af52af039af2b6ce"
< last-modified: Mon Sep 19 2022 07:06:41 GMT+0000 (Coordinated Universal Time)
# ...other headers

2nd call (or subsequent calls)

# request
curl -v --request GET \
  --url https://rntpzayauzmhsmxjjtfz.supabase.co/storage/v1/object/public/images/test-box-image.jpeg \
  --header 'if-modified-since: Mon Sep 19 2022 07:06:41 GMT+0000 (Coordinated Universal Time)' \
  --header 'if-none-match: "76d19e08653c1055af52af039af2b6ce"' \
  1> /dev/null

# response
< HTTP/2 200 
< cache-control: max-age=3600
< etag: "76d19e08653c1055af52af039af2b6ce"
< last-modified: Mon Sep 19 2022 07:06:41 GMT+0000 (Coordinated Universal Time)
# ...other headers

Expected behavior

Subsequent calls from client with etag and last modified header should return 304.

Screenshots

Refreshing images in Firefox (tested with curl and Chrome too) always gives 200, never 304.

See how there are 3 requests and all returns 200. The 2nd and the 3rd requests have matching etag and last modified, so it should return 304 instead.

System information

• OS: [macOS]
• Browser[chrome, firefox]

If we omit the if-modified-since header in the request, it returns back 304.

# request
curl -v --request GET \
  --url https://rntpzayauzmhsmxjjtfz.supabase.co/storage/v1/object/public/images/test-box-image.jpeg \
  --header 'if-none-match: "76d19e08653c1055af52af039af2b6ce"' \
  1> /dev/null

# response
< HTTP/2 304 
< cache-control: max-age=3600
< etag: "76d19e08653c1055af52af039af2b6ce"
< last-modified: Mon Sep 19 2022 07:06:41 GMT+0000 (Coordinated Universal Time)
# ...other headers

My current understanding is that last-modified is in the wrong date format. Unsure where this format comes from (s3 client or fastify?) but it does not follow the rfc2616 guideline on date format for last-modified header as it needs to be in GMT.

All HTTP date/time stamps MUST be represented in Greenwich Mean Time (GMT), without exception.

So

Mon Sep 19 2022 07:06:41 GMT+0000 (Coordinated Universal Time)

supposed to be

Mon, 19 Sep 2022 07:06:41 GMT

An open question from me: is requesting public image counted as a download for 'Storage egress'?

This is where I found out the first time about the possible bug, as I was checking my egress usage in my dashboard. If all the assumptions above are true (if it is indeed a bug and counted towards usages) then customers are overcharged with their egress usages.

Hi @ikhsan, I can confirm that your PR fixes the issue. The fix should be deployed to prod in a day or two.

Awesome, I'll test again in 2 days 👍

Hopefully, this prevents your customers (including me!) from overcharging storage egress

Hi @ikhsan this fix has been deployed to prod now. Thanks for reporting and the PR!

Glad that it works! Hopefully, this will make the storage egress calculation fairer 👍