Cache Policy for Server-Side-Website

[ndurchx]

There is an issue with the cache policy. If there happens multiple requests per second to an url with same paths but different get-parameters, it happens that you get cache hits from cloudfront and the request result is the same, regardless of the get parameters.

Solution: For the cache policy, all query strings have to be included into the cache key.

[mnapoli]

Thanks for the report. When we look at the CloudFront cache policy for Server-side website (for the backend):

https://github.com/getlift/lift/blob/b09781eabcee08fc9e59b30c1ca43ca4c85331bd/src/constructs/aws/ServerSideWebsite.ts#L127

the default cache is set to 0 seconds.

So the goal is that there is no caching by default (no matter the query parameters).

We could look into adding query string parameters to cache key, but first I'd love to clarify why you see caching happening:

• is it because your backend (code running on Lambda) returns caching headers? is that intentional?
• or could it be that defaultTtl: Duration.seconds(0) still results in caching happening for 1 second?

Any extra information you have is welcome.

[ndurchx]

The backend code does not send caching headers. It´s a echo + die single file script. The ttl=0 caching policy does not disable the caching for cloudfront. You see it at the response headers. Cloudfront delivers a cache:hit when multiple requests per second happen.

[bytestream]

@mnapoli I just ran into this and my experience is because the app is returning Cache-Control headers. This is documented on AWS, but not so clear from this libs perspective.

CloudFront uses this setting’s value as the object’s TTL only when the origin does not send Cache-Control or Expires headers with the object. https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/controlling-the-cache-key.html?icmpid=docs_cf_help_panel#cache-key-understand-cache-policy-ttl

How do I specify that I want the cache policy to include All query parameters via serverless.yml ?

---

Based on https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/using-managed-cache-policies.html I note one critical difference between the default BackendPolicy created by this lib and AWS's CachingDisabled policy:

- Maximum TTL: 0 seconds
+ Maximum TTL: 31536000 seconds

But I note if I change the default policy to 0 then it produces an error in AWS console: The parameter HeaderBehavior is invalid for policy with caching disabled.

https://github.com/getlift/lift/blob/7b1a3f76522eef7bfaba311c1860bf531b9eb0fd/src/constructs/aws/ServerSideWebsite.ts#L128 suggests the HeaderBehaviour is necessary, despite the goal being to disable caching which seems somewhat contradictory?

[mnapoli]

Hey, we've had another report of this behavior here: https://github.com/brefphp/bref/discussions/1200

I'm not +1 to change the cache policy to disable cache entirely (Maximum TTL: 0 seconds), as you suggest. If that means removing the "HeaderBehavior" config, why not? 🤷

Would you have time to send a pull request for this change?

[mnapoli]

Or wait, we could actually switch to the official "CachingDisabled" policy then.

[jjmontgo]

I'd like to suggest not using the "CachingDisabled" policy.

While the application I'm using doesn't need the Authorization header, I know JWT uses it, and I've had callback pages that needed to process the Authorization header.

So instead I tried ndurchx's original suggestion, and I used the custom caching policy with All Cookies and All Query Strings in the cache key.

That worked. Same-second requests did not result in a cache hit, and the Authorization header was still passed in.

[mnapoli]

Update: I looked into this problem again and tried a few things, including #218.

1. Switching to the "CachingDisabled" policy drops the Authorization header, we indeed don't want that -> not a good solution
2. Adding all query strings and all cookies to the cache key is IMO an incomplete approach: we don't want to cache based on cookies or query strings, we don't want cache at all. -> not a good solution
3. Adding minTtl: 0 doesn't change anything -> not needed
4. Adding maxTtl: 0 doesn't work, it's equivalent to the "CachingDisabled" policy above and we run into the Authorization header problem -> not a good solution
5. CloudFront seems to cache requests in the same second when no cache headers are set

Regarding 5, to clarify: I used to believe that CloudFront might cache responses that contained any kind of cache headers (whether it was intentional or not). But after testing @ndurchx's scenario (simple "echo Hello world" PHP script that doesn't set any header), I do get cache hits here too.

My conclusion so far is that Lift's current config is not working as expected. It may be a CloudFront bug, but I don't want to hold my breath on AWS fixing this.

How can we fix this? #218 will not fix the problem.

The only solution I could think of is this one, but it is very convoluted:

• Whitelist specific headers via a custom Origin policy. Do NOT whitelist the Host header. Downside: we can whitelist up to 10 headers.
• Use the official CachingDisabled policy (no caching).
• We cannot forward the Authorization header by whitelisting it (https://stackoverflow.com/a/66619476/245552).
• Forward the Authorization header in a custom X-Bref-Authorization header via a CloudFront function.
• In the Bref runtimes, set the X-Bref-Authorization header to the Authorization header.

Any other idea?

[mnapoli]

I tried another approach:

Skip CloudFront, use API Gateway + S3.

• Host the website behind API Gateway
• Serve assets from the S3 public URL
• Set up a custom domain with API Gateway Edge endpoint to get the HTTP -> HTTPS redirect

Conclusion: this works. No cache hits (it seems that caching is entirely disabled in the "Edge distribution"), and all headers are forwarded, including the Authorization header.

Downsides:

• Assets are hosted on a different domain.
• We cannot use CloudFront functions to redirect to the main domain.
• When no custom domain is set up, REST APIs have a /prod URL prefix. We may use Lambda Function URLs when no domain is configured.

[t-richard]

Downsides:

• Assets are hosted on a different domain.
• We cannot use CloudFront functions to redirect to the main domain.
• When no custom domain is set up, REST APIs have a /prod URL prefix. We may use Lambda Function URLs when no domain is configured.

Those are some nice features to give up on :frowning: But if that's the only realistic solution in a prod environment, we may be forced to switch...

We may use Lambda Function URLs when no domain is configured.

That can work with Bref, but in other cases it may not work because API gateway v1 and v2 payloads are different (function URLs are using v2 format).

[mnapoli]

Some more updates: turns out there are 2 different behaviors into play with CloudFront:

• response caching
• request collapsing

The problem we are seeing is actually not related to caching, but to request collapsing. So it's not a matter of "caching below 1 second". It's just that CloudFront can consider 2 identical requests as the same and will return the same server response.

More resources about that:

• https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/RequestAndResponseBehaviorCustomOrigin.html#request-custom-traffic-spikes
• https://repost.aws/questions/QUe8vbIDvNTiy8ZSYDv85iTQ/amazon-cloud-front-collapsed-request-answered-with-cache-control-no-store
• https://stackoverflow.com/questions/67313451/problem-with-caching-in-cloudfront-cdn-aws-cache-and-collapse-forward-are-ac
• https://stackoverflow.com/questions/69421737/refreshhit-from-cloudfront-even-with-cache-control-max-age-0-no-store/69455222#69455222

Request collapsing is not dependent on HTTP response cache headers. CloudFront decides to "collapse" similar requests before even getting any response.

That means the only way to prevent that entirely (which we want) is to disable CloudFront caching.

I have a call with someone from the CloudFront team soon, I'll try to find out more. In the meantime, I will probably reopen and merge #218 because it avoids the problem in probably 99% of cases. It's good mitigation.