zarr-python
zarr-python copied to clipboard
[v3] First step to generalizes ndarray and bytes
This PR introduces two new classes NDBuffer and Buffer to represent the data argument between components.
Currently, we use numpy.ndarray and bytes to pass around data between components. As discussed in #1751, it would be good with a generalization of the data containers to enable other memory types.
As a first step, NDBuffer and Buffer are backed by numpy arrays. In follow-up PRs, we can discuss/implement alternative backends such as cupy arrays or pytorch tensors. The important point here is to agree on an abstraction that can facilitate a broad range of array/data types.
cc. @akshaysubr, @jakirkham