RFC Caching

[ahopkins]

Background

Sanic has two opposing goals: speed and flexibility. While these goals often both strive for simplicity, they also are often competing in design. To handle a wide variety of scenarios, the server must run through a number of checks, parsing data from the request, etc.

I propose a new opt-in caching mechanism that can be configured (much like strict_slashes or host) at various levels, in order of precedent:

• route,
• blueprint, and
• application.

There are potentially different places that we can serve the cache from:

• after reading the request line (fastest time to response, but would ignore incoming headers and middleware)
• after reading headers (would ignore middleware)
• after retrieving the route (we now have access to the path, params, etc)

This last one potentially gives the developer the most opportunity to decide how to cache a request. At its essence, the idea would be to capture the bytes being sent in the response to a key/value store, and serve those bytes back instead of executing the handlers. With some initial crude testing, this gain can be 15-20%.

API

I'd suggest that the developer can control the caching in several ways:

app = Sanic("CachedApp", cache=RouteCache.PATH)

app.config.ROUTE_CACHE = "headers"

bp = Blueprint("NoCaching", cache=None)

app.route("/super-cache", cache=RouteCache.RAW)(...)

bp.route("/super-cache", cache="params)(...)

As you can see, I intentionally am mixing Enum with strings.

Caching strategies to include:

• RouteCache.NONE - Same as None
• RouteCache.PATH - Use the parsed path from the router
• RouteCache.PARAMS - Use the parsed path and params from the router
• RouteCache.HEADERS - Use the parsed headers
• RouteCache.RAW - Use raw request line

We potentially can give the developer some more control to add caching functions, but I worry that would be very time consuming on the retrieve side. Perhaps we could allow some customization to cache (for example) based upon a specific header.

Potential strategies

I think that the caching layer would be easy to add inside the new router, as a composable layer. By default we do nothing, therefore no performance change. When a route is cacheable, we inject the cache storage function into the handler, and also the appropriate location in the request/response cycle.

Prebuilt responses

Ultimately the idea is to build a set of cached responses that can be sent to the client early without execution. In certain circumstances, these responses might be able to be built at startup (or, rebuilt at some point during the running of the server). Therefore, we could also provide a mechanism to allow for responses to be generated for select cached requests before the first request ever arrives.

[sjsadowski]

I will weigh in here that in my opinion, flexibility should always be secondary to speed in the decision-making process. It shouldn't be discarded completely, but the trade-offs should be weighed appropriately,

With that being said, adding caching to increase responsiveness of the handlers is a good thing in my opinion and if we're making it completely optional so that the default behavior doesn't change (at least not until further down the road), this gets an upvote from me.

[ahopkins]

As a point of clarification, I agree. Speed is a primary concern that should generally trump others. However, Sanic should not become an opinionated framework, and should ultimately provide the developer the greatest amount of flexibility possible.

[sjsadowski]

Absolutely, but feature decisions will always carry a bias - I don't think that would make the framework opinionated, but would reflect our choice of implementing things. For this reason, I think that bias should lean toward speed, and I think caching is a good item for that.

[SkBlaz]

Hi! First, great work with the library. Any idea what preferred caching solution currently is? (few years later?)