kerchunk
kerchunk copied to clipboard
fsspec
Cloud-friendly access to archival data
What are the required dependencies of kerchunk? Right now https://github.com/fsspec/kerchunk/blob/6b57f0620c9e1e55206523ac54dadf60830b0935/requirements.txt has ujson on fsspec. Working with HDF5 files (https://github.com/fsspec/kerchunk/blob/main/kerchunk/hdf.py) seems to require h5py, zarr (and numcodecs), and numpy. combine requires...
Current references for, e.g., nwm-forecast refer to paths like noaa-nwm-pds/nwm.20210701 , but (as of today) the oldest folder in the bucket is noaa-nwm-pds/nwm.20211128 cc @rsignell-usgs
Currently, we include only the URLs of references in the generated JSON files. With https://github.com/fsspec/filesystem_spec/pull/867 , this becomes more complicated, as we can have multiple remote filesystems, each with their...
How does this compare to what STAC is trying to achieve? Or maybe this is a tool which can work together with STAC? A quick search in your documentation and...
For some reason I have files (model output) where the `hdf5` `fillvalue` property is different from the `_FillValue` attribute: ```python f = h5py.File("", mode="r") f["latitude"].fillvalue != f["latitude"].attrs["_FillValue"] # significantly different...
I'm trying to kerchunk the medium range netcdf files from the National Water Model, which are only accessible via HTTPS from NOMADS (not S3). I can successfully a few files...
https://observablehq.com/@manzt/ome-tiff-as-filesystemreference @manzt , perhaps you want to comment on the status of your zarr storage driver.
MultiZarrToZarr expects a collection of Zarr ReferenceFileSystem JSON paths. What steps are required to generate ReferenceFileSystem JSON from a Zarr Directory or Zip and is this something for kerchunk to...
So I'm going to start trying to put together a way to get reference maker to work with the output from the model I use (https://github.com/RAMSmodel/rams). This is probably a...
Following [our discussion in dask #8045](https://github.com/dask/dask/issues/8045) I’m seeking to contributing a CSV module to `fsspec-reference-maker`, and wanted to kick off a feature proposal issue here to clarify some aspects. Hope...
