Akshay Subramaniam

We are working on a paper right now and should put it up on arxiv in the next few weeks. Stay tuned!

This discussion about how to orchestrate a producer-consumer workflow between chunk readers (Store) and decompressors (Codecs) is really interesting. But I worry that it is taking away from the more...

> There is a bit of design work to be done in terms of figuring out the best way to implement this feature. It might be helpful to have a...

Wanted to update here that an implementation of a `Codec` with batched encode and decode methods is now merged into kvikIO: https://github.com/rapidsai/kvikio/pull/249 Here's the basic structure of the API: ```python...

@rabernat Thanks, I'll work on a PR for it then. Might have some questions about packaging which I'll post here if that's okay.

If the abstract class is not required in Python, looks like moving `f90wrap_shapes__get__shapetype(this, shapetype)` to classes that extend the abstract class is sufficient. Maybe a pre-processor that copies everything in...

> It is going to hard to support compression that are compatible with the ones built into HDF5. This is not necessarily true. It would be if using the nvCOMP...

Agreed, this would indeed be very valuable!

This PR adds this for the batched codecs using the nvcomp LLIF: https://github.com/rapidsai/kvikio/pull/249

> say "lz4" codec, but "gpu" version Agreed! How we can achieve this though? Maybe a new `numcodecs.codecs.gpu` entrypoint and a `device` entry in `numcodecs.registry.get_codec`?