Respect HTTP Cache headers

[evert]

At the moment we simply cache everything coming from a server, and only expire this when refresh is called. I'm not 100% certain if this is the right way to go about things.

It's worth researching this a bit better.

[evert]

Not making this a 6.0 goal (again). It hasn't really been asked for too much and it's too large.

[crabmusket]

Posting this in case it's helpful down the road: https://github.com/kornelski/http-cache-semantics

[evert]

Thanks for the link. Looks like that could do a lot of the heavy lifting that I didn't feel like doing :D

[sazzer]

Being able to disable any caching that Ketting has built-in, and just fall back on the browser cache, would be a simple way to get correct cache behaviour for relatively little work. (Though obviously that won't work for caching embedded resources or anything clever like that)

[evert]

Yeah, it has a bunch of issues... off the top of my head:

1. No way to programmatically expire
2. No support for Link: <..>; rel="invalidates" which is really just a variant of 1.
3. No way to store things in the cache for embedded items, PUT
4. No support for Prefer: return=representation
5. Can't trigger event handlers like the 'stale' and 'update' events.

So that approach works, but only really if you're using a simpler subset of ketting's features.

[sazzer]

Fair enough - I didn't realise quite how much of Ketting depended on the caching stuff!

[evert]

Yeah, cache has become increasingly important =) Especially since Ketting 6. When this ticket was made it was pretty early days for this lib.

[evert]

So yea, I don't have a great idea yet for how to do this proper =)

I want better ways for a server to tell Ketting to do things like 'never cache', and the obvious header is Cache: no-store, which is basically the default for most APIs, but the issue is with the opposite of this..what if we want the server to store an object for an hour? That works with the usual headers, but then Ketting has a few different ways (like mentioned above) where it knows it needs to expire that cache early, or it actually knows the new representation..

So all of this is tricky to handle.

For now I think I'm just going to stick with the Ketting cache as-is, and use the browser cache as an additional cache layer for cases where we know for sure we won't need a new fetch until later.

And maybe come up with a ketting-specific cache header that's ignored by a browser but uses similar semantics. Server-controlled caching is so powerful! The HATEOAS design basically means we want to put as many decisions as possible on the server, and have the client make very few decisions about this kind of stuff.

[sazzer]

Ketting specific headers requires the server author to know/assume that the client is using ketting. Which implies that the two are connected somehow. Things like CORS allows me to write a client for some server that I'm in no way connected to - e.g. I could write a custom client to the Google APIs.

On Mon, 30 Nov 2020, 18:59 Evert Pot, [email protected] wrote:

So yea, I don't have a great idea yet for how to do this proper =)

I want better ways for a server to tell Ketting to do things like 'never cache', and the obvious header is Cache: no-store, which is basically the default for most APIs, but the issue is with the opposite of this..what if we want the server to store an object for an hour? That works with the usual headers, but then Ketting has a few different ways (like mentioned above) where it knows it needs to expire that cache early, or it actually knows the new representation..

So all of this is tricky to handle.

For now I think I'm just going to stick with the Ketting cache as-is, and use the browser cache as an additional cache layer for cases where we know for sure we won't need a new fetch until later.

And maybe come up with a ketting-specific cache header that's ignored by a browser but uses similar semantics. Server-controlled caching is so powerful! The HATEOAS design basically means we want to put as many decisions as possible on the server, and have the client make very few decisions about this kind of stuff.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/badgateway/ketting/issues/21#issuecomment-735978043, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAQEGH35WH4K6X5KZ7XLLTSSPTSVANCNFSM4DIJMTEQ .

[evert]

You are absolutely right @sazzer , it's a solution that only works if you can control server & client. Of course there's nothing stopping people from using these headers for other servers & clients.

In cases where you can't do anything server-side, you're always going to have to add more logic to the client-side. But, if the server does have these bonus headers they shouldn't affect semantics of clients that don't understand them, but clients that do can potentially do smarter things.