Only send up-to-date control header for responses with short cache timeouts

[KyleAMathews]

Right now I believe we're sending an up-to-date control message at the end of every log (correct me if wrong).

This is problematic as the default cache timeout is 60 seconds. So someone could be as much as 59 seconds stale while thinking they're live.

A potential pattern for solving this:

1. -1 offset gets long cache value to optimize cache hits for bulk of data e.g. even 1hr.
2. then vary the length of the cache timeout based on how old the offset on the request is. E.g. if the offset is 30 minutes out of date, set the cache to 30s. If the offset is < 10m then set the cache to 1s.
3. return up-to-date if the offset is < 10m or if there's nothing newer than the offset.

This would mean for most initial syncs, we'd do 2 (and occasionally 3) requests to get to up-to-date. Since they're all from the CDN, it'll be pretty fast. And even with a 1s cache timeout, the CDN will still handle 99% of the load as with stale-while-revalidate, the CDN is the only one going back to the origin to get new data.

[icehaunter]

Does this imply keeping track of a timestamp for last-seen offset?

[KyleAMathews]

It would. Unless of course we come up with another scheme to accomplish the same issue. The offset could have the time embedded e.g. the number of 15 min segments or something since midnight would suffice for our needs.

[msfstef]

This raises a more general issue - the same chunk of the log might contain an up-to-date message or it might not, but it gets assigned the same etag and will be treated as the same chunk of data cache-wise.

One easy workaround that moves the responsibility to the client would be to include a timestamp in the up-to-date control message - that way even if a cached chunk is retrieved by the client they examine the timestamp on the up-to-date message, which was assigned by the original server response, and decide whether they want to treat it as out of date (client-configurable if we want). I think this might be good to have regardless.

Other than that I do think that we might want a layer for determining cache control headers for chunks served depending on various factors (is it a "full" chunk? is it an up-to-date chunk? etc) I'm happy to pick this up.

[thruflo]

I think we need to carefully look at this.

[KyleAMathews]

@msfstef and I chatted a bit about this in private chat — the basic idea is to figure out an algorithm that balances freshness and cache hits. Reduce the server work to update caches in http proxies to a bare minimum. Also that clients that don't want to hit the live endpoint have a guarantee around how fresh the cache is.