zarr-python icon indicating copy to clipboard operation
zarr-python copied to clipboard

Proposal: Make Obstore backend the default for s3/gcs/azure/https

Open jhamman opened this issue 2 months ago • 1 comments

The Obstore backend in Zarr for about 6 months. Its working great and is significantly more performant that the fsspec alternatives. See maxrjones/zarr-obstore-performance for performance benchmarks. With this in mind, I'm proposing that we switch to using Obstore as the default in Zarr-Python if is available. Fsspec would become the fallback for s3/gcs/azure/https and would remain the default for all other remote stores.

Image

jhamman avatar Oct 14 '25 11:10 jhamman

+1, those numbers are hard to argue with.

This will probably surface some long-tail edge-case bug/change in behaviour, so we should add a configurable option.

dcherian avatar Oct 14 '25 13:10 dcherian