kerchunk
kerchunk copied to clipboard
fsspec
Cloud-friendly access to archival data
https://github.com/cgohlke/tifffile provides a way to get kerchunk references out of individual TIFF files and sometimes sequences (see long thread: https://github.com/cgohlke/tifffile/issues/125). It also provides access to TIFF tags, which, for the...
An example of an hdf5 file format. https://geoh5py.readthedocs.io/en/stable/ "The geoh5py library has been created for the manipulation and storage of a wide range of geoscientific data (points, curve, surface, 2D...
Currently, translating HDF5 to Zarr will result in a Zarr store with identical chunks as the source. If the source isn't chunked, this will cause *worse* performance when you slice...
I am very new at this, so I might not comprehend the exact goal of `scan_grib` but here we go (this is a follow up from [discourse](https://discourse.pangeo.io/t/accessing-grib2-files-as-a-single-cloud-friendly-dataset-in-xarray-through-kerchunk/2401/12)): **What's the issue**:...
I have input zarr datasets which feature identical chunk sizes except for the unlimited dimension, which is time. Each of my input datasets has trailing chunks which differ from one...
I have a consolidated JSON with a "best time series" dataset that reads from a collection of HRRR grib2 files on S3. The data access and visualization of data in...
- I want to use `kerchunk` to open climate model netcdf output stored 1 or 12 timestep/s per file in many files as one huge `zarr`. - I do not...
I'm trying to combine three netcdf file jsons using the following notebook: https://nbviewer.org/gist/6117f63ba1b266ea713249b98492ce76 (which could be reproduced if you have AWS credentials -- it reads from a requester pays bucket)...
Hi, I'm new to kerchunk. I followed the example (see below) to convert the NWM S3Files to Zarr. It worked well for files before 2007. After 2007, I got an...
The folks at CEDA have come out with a new standard that aims to solve the same problem as kerchunk for the many-netcdf use case: - https://techblog.ceda.ac.uk/2021/12/08/cfa-conventions-intro.html - https://github.com/NCAS-CMS/cfa-conventions The...
