Tom Nicholas

Results 303 issues of Tom Nicholas

Zarr-python has a bot which automatically labels PRs as needing release notes if they [don't contain any changes](https://github.com/zarr-developers/zarr-python/blob/main/.github/labeler.yml) to the release notes docs file. I want that.

enhancement
CI

Follows on from #340

We want to be able to create, combine, serialize, and use manifests that point to very large numbers of files. The largest Zarr stores we already see have `O(1e6)` chunks...

performance

This library is supposed to have an API so small it's practically non-existent (everything being done through xarray instead), but we still probably need documentation that's more than just the...

documentation

Separate the idea of v2.0.0 (meaning feature-completeness without zarr spec changes) from longer-term improvements that require new ZEPs.

documentation

We should answer that question in the FAQ docs, partly using an abridged version of the explanation in Matt's excellent blog post https://matthewrocklin.com/blog/work/2018/02/06/hdf-in-the-cloud cc @ianhi

documentation

We can't support general indexing into ManifestArrays without loading data values, but we could imagine supporting slicing, so long as the slices align with chunk boundaries. The implementation would just...

enhancement
zarr-specs

HTML reprs are cool. We could make one for `ManifestArray`, by stealing the code used for the `dask.Array` (or `cubed.Array`) repr. It might help with visualising the chunking pattern, and...

enhancement

We should support generating references from files containing multiple groups in the same way that `xr.open_dataset` and `datatree.open_datatree` work. So we should add a new `open_virtual_datatree` function, and a new...

enhancement

@rabernat made the interesting point to me that uncompressed files (e.g. netCDF3 files) have no specific chunking, as you can start reading bytes from any point in the file immediately,...

enhancement
zarr-python
Kerchunk