Use Surrogate-Control across all backends instead of custom X-Reverse-Proxy-TTL

[andrerom]

As discussed in the original Fastly PR in order to generalize TTL handling across backends like Varnish, Fastly, Symfony, it could be an idea to:

1. abstract it on client side?
2. to be able to emit Surrogate-Control for fastly
3. ~If feasible look into also emit Surrogate-Control on Varnish instead of X-Reverse-Proxy-TTL, VCL could still handle for X-Reverse-Proxy-TTL for BC.~

https://www.w3.org/TR/edge-arch/

[dbu]

i assume its more complicated than allowing to change the header name, as surrogate-control will need some additional thing to tell that the number is the ttl on that cache? there seems to be discussions on how varnish itself should support the surrogate-control header: https://github.com/varnishcache/varnish-cache/issues/2893

making the thing more flexible definitely sounds like a good idea. for varnish, we might want to wait for them to decide how to handle it, to avoid having our custom thing that is different from how varnish works. (there is also stuff about handling surrogate names so that the application could target only some of the surrogates and whatnot. not necessarily something we have to support right away, but something to keep in mind with the solution that we come up with.)

i guess ttl should move out of the cache control listener, or be injected as a strategy so we can have the current behaviour or the fastly one or maybe a future thing for varnish with surrogate-control.

[andrerom]

how varnish itself should support the surrogate-control header: varnishcache/varnish-cache#2893

After having read that, I guess it's indeed best to drop the 3 item for now.

The main needs are 1. is to be able to not use s-maxage for ttl across our supported proxies, to leave that for normal internet proxies if people want, and 2. an abstract way to do it so there is one way to set TTL across all supported proxies, in case of Fastly emit Surrogate-Control header.

Adds up on your side? :)

[dbu]

Yep, fully agree!

[dbu]

after discussion at https://github.com/symfony/symfony/pull/41369 , an additional thing to think about is that the Age and Date headers prevent max-age and s-maxage to work as (likely) intended when setting a custom TTL. does the surrogate-control header define anything about that?

[andrerom]

Depends on what you mean, but I can't see any mention of Age here: https://www.w3.org/TR/edge-arch/#:~:text=2.2%20Surrogate-Control%20Header

But indeed seems this is a known issue when letting reverse proxy cache for long time: https://book.varnish-software.com/3.0/HTTP.html#age

Ideally browsers should have some heuristics to prefer other headers then Age if set, but possible fix for this is unsetting Age in VCL, or rename it so you can still see age of cache for debugging issues.

EDIT: Or other client headers should be used here, ref question if stale headers might work better on the symfony PR.

[alexander-schranz]

Find this very interesting Topic! Do we know if Surrogate-Control is used in other cache service like CloudFront, Cloudflare, ...?

I found some more resources about this topic about another header <targeted>-cache-control header:

• https://www.akamai.com/blog/news/targeted-cache-control
• https://datatracker.ietf.org/doc/html/draft-cdn-control-header-01 (expired? draft)
• https://developers.cloudflare.com/cache/about/cdn-cache-control/

Requested the state of the CDN-Cache-Control via Twitter here: https://twitter.com/alex_s_/status/1557679407531171843 some interesting links from fasly yet. They currently supporting Surrogate-Control but not yet the CDN-Cache-Control.

[dbu]

oh, that twitter discussion looks quite interesting. if we add flexibility in this regard, adding Surrogate-Control support additionally seems not wrong and probably not too complicated. but the -cache-control draft looks promising. your tweet only got responses from fastly until now... would indeed be interesting to know if the draft can make it into a proper RFC... implementing something that might still change when the RFC is formalized is a bit unfortunate, but with the necessary disclaimers i think it should be ok to do it.

i wonder if the symfony httpcache itself should consider itself a CDN and implement this as well? possibly also registering SymfonyCDN-Cache-Control? (but that discussion is out of scope here)

[alexander-schranz]

Cloudflare did response with support for the CDN-Cache-Control did got lost in my notifications: https://twitter.com/jgrahamc/status/1558142754034794501

The rfc called targeted-cache-control seems also be published two month ago as RFC9213: https://datatracker.ietf.org/doc/rfc9213/

Cloudflare docs: https://blog.cloudflare.com/cdn-cache-control/

[dbu]

ah, that is excellent news! lets open a new issue to implement RFC-9213 and deprecate x-reverse-proxy-ttl then! we will need to make the header name configurable because it could be a vendor-specific name - when we do that, we can simply allow to specify "Surrogate-Control" as well to cover this story. (though i read the twitter thread as saying that fastly will eventually support the new RFC 9213 as well)

are you interested in working on that?

[alexander-schranz]

I opened an issue in the Symfony Core: https://github.com/symfony/symfony/issues/47288 I think it would be great if the HttpFoundation component allow us easier set and manipulate this headers.