zarr-python
zarr-python copied to clipboard
Proposal: Make Obstore backend the default for s3/gcs/azure/https
The Obstore backend in Zarr for about 6 months. Its working great and is significantly more performant that the fsspec alternatives. See maxrjones/zarr-obstore-performance for performance benchmarks. With this in mind, I'm proposing that we switch to using Obstore as the default in Zarr-Python if is available. Fsspec would become the fallback for s3/gcs/azure/https and would remain the default for all other remote stores.
+1, those numbers are hard to argue with.
This will probably surface some long-tail edge-case bug/change in behaviour, so we should add a configurable option.