electric icon indicating copy to clipboard operation
electric copied to clipboard

Only send up-to-date control header for responses with short cache timeouts

Open KyleAMathews opened this issue 6 months ago • 5 comments

Right now I believe we're sending an up-to-date control message at the end of every log (correct me if wrong).

This is problematic as the default cache timeout is 60 seconds. So someone could be as much as 59 seconds stale while thinking they're live.

A potential pattern for solving this:

  1. -1 offset gets long cache value to optimize cache hits for bulk of data e.g. even 1hr.
  2. then vary the length of the cache timeout based on how old the offset on the request is. E.g. if the offset is 30 minutes out of date, set the cache to 30s. If the offset is < 10m then set the cache to 1s.
  3. return up-to-date if the offset is < 10m or if there's nothing newer than the offset.

This would mean for most initial syncs, we'd do 2 (and occasionally 3) requests to get to up-to-date. Since they're all from the CDN, it'll be pretty fast. And even with a 1s cache timeout, the CDN will still handle 99% of the load as with stale-while-revalidate, the CDN is the only one going back to the origin to get new data.

KyleAMathews avatar Jul 25 '24 20:07 KyleAMathews