Support access to raw encoded chunks
I think we need to start thinking about how to let the client elide
read -> decompress -> deserialize -> serialize -> compress -> transport -> decompress -> deserialize
into
read -> transport -> decompress -> deserialize
if the client happens to understand the formats and codec(s) that the data happens to be stored in.
This involves:
- Accessing a description of the raw encoding, perhaps via a new
raw_encodingproperty on Adapters. We may take inspiration from Zarr regarding how to spell this. - Comparing a requests's
AcceptandAccept-Encodingheaders to the raw encoding, and short-circuiting the serialization and (sometimes) the compression process if the client can accept the raw encoding.
I think there is a clear path to doing this in a backward-compatible way, so I have removed it from the v0.1.0 milestone.
This could also work for nodes. If I ask for a node that happens to be an HDF5 file as HDF5, we can just spool the HDF5 file.
It might make sense to give the client a way to know this by sticking some information in meta.