parcels icon indicating copy to clipboard operation
parcels copied to clipboard

Choice of Zarr compressor

Open VeckoTheGecko opened this issue 6 months ago • 4 comments

The writing of the zarr file in particlefile.py doesn't look to set a compressor. Setting a compressor can significantly help trading off compute for storage or vice versa [1]. The default chosen by zarr is the Blosc compressor, which is a "meta-compressor" (using different algorithms under the hood). Perhaps its worth investigating other compressors to see if there's one that is best suited for our simulation output.

@erikvansebille do you know if zarr compressors have been a topic of discussion before?

VeckoTheGecko avatar Aug 20 '24 16:08 VeckoTheGecko