kerchunk
kerchunk copied to clipboard
fsspec
Cloud-friendly access to archival data
The current [Kerchunk documentation](https://fsspec.github.io/kerchunk/index.html) is minimal in a good way and the content to the point. Would you, however, see value added to refresh its look, feel and interactivity? Both...
New to kerchunk and encountered this issue running the MSFT Planetary Computer example [notebook](https://github.com/microsoft/PlanetaryComputerExamples/blob/main/datasets/nasa-nex-gddp-cmip6/nasa-nex-gddp-cmip6-example.ipynb). Simply running twice cells 13 & 14 that read and plot point single variable time-series for...
While experimenting with kerchunking some Icesat2 ATL08 data I noticed an issue where using `MultiZarrToZarr` with non-dimension coordinates that had partial chunks resulted in empty values for those variables in...
Allow kerchunk based concatenation of zarr arrays with variable length chunks. This is mostly to allow me to play around with some downstream usecases. This is basically feature complete at...
We've created some references for NetCDF3-64bit-offset files, which we need to do locally (since we can't access them from object storage). So to convert the local `combined64.json` to point to...
When I try to run `SingleHdf5ToZarr` on an HDF5 file with a scalar HDF5 dataset that has a variable length utf-8 string dtype or a variable length ascii bytes dtype,...
https://docs.xarray.dev/en/stable/internals/how-to-add-new-backend.html describes how to add a new backend to xarray's open_dataset. Adding a backend will make it easier for users to consume kerchunk references. Here's a prototype: ```python from xarray.backends...
This feels like its probably an edge case, but I wanted to bring it up in case there's an opportunity to help fix this. I'm attempting to read a GZIP...
@tomnicholas and I noted @martindurant's mention in https://github.com/fsspec/kerchunk/pull/17#issuecomment-789205972 of the possibility of eventually representing the [references spec](https://fsspec.github.io/kerchunk/spec.html) as JSON Schema. We do not see further mention of JSON Schema in...
### Problem Kerchunk user code currently passes around an obscure multiply-nested "reference dict" object. This is hard to read, interrogate, validate, or reason about. ### Suggestion Instead create a new...
