Rest icon indicating copy to clipboard operation
Rest copied to clipboard

Implement server-side response cache

Open nickl- opened this issue 13 years ago • 7 comments

This is a topic I've been burning to raise but there are still features required that takes a higher priority than this. Since the RESTful API architecture involves extensive CACHE considerations there is not much need to further motivate the benefit of having a properly implemented CACHE solution at the end-point, aside from the obvious performance and load shedding advantages, to be able to verify and test conformance of the developed REST implementation without the need for external CACHE providersp

It is only until now that we are discussing conditional requests that this has become pertinent as the framework will see a huge benefit if it is able to cache and make request provisioning decisions based on cached variants rather than re-requesting the Routers to populate a response on every request. for state information.

2616 allows for ETag to be used as an "opaque" identifier instead of dates:

The ETag response-header field value, an entity tag, provides for an "opaque" cache validator. This might allow more reliable validation in situations where it is inconvenient to store modification dates, where the one-second resolution of HTTP date values is not sufficient, or where the origin server wishes to avoid certain paradoxes that might arise from the use of modification dates.

Although I this will not be sufficient as the only identifier and we will benefit from having an additionally indexed key when ETag is not available, based on a calculated checksum of the following HTTP artifacts: (Of the top of my head this will have to be verified)

  • location

  • accept header

  • method

    In the interim it is safe to speculate that we would consider plug-able cache provisioning utilizing some/all of the following :

  • via file cache - simple and free from dependencies utilizing PHPs file_put/file_get_contents();

  • via PDO as database driven cache

  • via APC provisioned

  • via REDIS provisioned

  • via Memcache provisioned

  • via APC provisioned

  • via Solr provisioned

  • via ElasticSearch provisioned

  • via Reverse Proxy (Squid probably) provisioned (not sure if this is exactly what we have in mind though.

Aside from the obvious requirement that the design should be flexible to easily cater for additions to this list of providers I would additionally see the functionality being transparently exposing an API that can be utilized by other components of the application where this functionality may be required without impacting the core cache.

What are your thoughts?

nickl- avatar Jun 22 '12 19:06 nickl-

eZ Component's Cache library might be worth a look at.

features: ezcCacheManager - This is the optional manager - will store only the configurations by default, only instantiating the cache object when requested. ezcCacheStack - multiple cache storages can be combined using a technique called "hierarchical caching". ezcCacheStorageFilePlain - utilizes the file_get_contents() and file_put_contents() PHP functions. ezcCacheStorageFileArray - can store array structures and will keep PHP data types intact. ezcCacheStorageFileEvalArray - uses PHP's eval() method instead of require() to restore the cached data, the stored data will not be cached again in PHP accelerators like APC. ezcCacheStorageMemcachePlain - uses the PHP Memcache extension to store cache objects in memory. ezcCacheStorageApcPlain - uses the PHP APC extension to store cache objects in memory. ezcCacheStorageFileApcArray - uses the PHP APC extension to store array structures in memory.

nickl- avatar Jun 22 '12 20:06 nickl-

I would go for something much more simple, similar to what I proposed on #40.

$r3->any('/images/*', 'MyImageHandler')->etag(true, function($data) {
    $tag = $data->myAbsolutelyUniqueId+serialize($headers); //custom logic
    return function($output) use ($tag) {
        MyCacheLib::store($tag, $output);
    };
});

Would probably use something like a new ProxyableOutput interface, similar to ProxyableThrough but for the entire output, not only the callback returns. Two closures are not that complex, yay? Also not sure about the closure params. Seems that these ones are enough.

alganet avatar Jul 03 '12 02:07 alganet

@alganet You mentioned StreamWrappers in passing to utilize for this. There was also a link to another project with a stub of a StreamWrapper implementation you did.

On both of these I am not entirely clear, would you elaborate please, don't forget the link?

nickl- avatar Aug 10 '12 11:08 nickl-

@nickl- can't remember why I mentioned streams!

alganet avatar Aug 26 '12 15:08 alganet

@alganet You had a skeleton interface for stream wrappers committed somewhere, I looked and I can't find the logs or issue anywhere.

You had the idea to do processing ie md5/ETAG/caching via the wrappers, which sounds awesome but I only started thinking about it more at a later stage and I'm not entirely sure I didn't miss something else which is why I want to pick your brain on this again.

Ring any bells now?

nickl- avatar Aug 27 '12 06:08 nickl-

Maybe I was wondering about auto-generated ETags for stream resources. I really can't remember, maybe I need to drink less =(

alganet avatar Aug 30 '12 14:08 alganet

Or it's time for another drink XD

nickl- avatar Aug 30 '12 15:08 nickl-