Tom Nicholas
Tom Nicholas
Also could we abstract this away for the user? So that developers of new backends have to understand `obstore` but people calling `open_virtual_dataset('known_filetype.nc')` don't?
> If `store=None` we would call `default_object_store()` to infer the protocol and prefix from the filepath, then create the relevant store instance. I realised I now need something like `default_object_store()`...
> I can take a go at building this today. No pressure to do it today! Would be useful to have during the latter half of next week though. >...
Hi @darothen - although we're planning to release v2.0 this week, that isn't slated to include GRIB support. However @sharkinsspatial has been working on a GRIB parser very recently, so...
Very interesting @frazane ! Is EarthKit able to export the byte ranges? Do you have any code to share?
Whilst this passes the CI tests, it doesn't work at scale for this weird reason that I don't yet understand - see https://github.com/lithops-cloud/lithops/issues/1429.
> this lithops failure Umm that's weird. The CI is using lithops `3.6.0` - what are you using on your machine? (Though it worked on mine too ofc and `3.6.0`...
> replace the kerchunk parser's use of fsspec's [LazyReferenceMapper](https://github.com/fsspec/filesystem_spec/blob/66340a15ef6a11aa89b4f23a4370c53316d85362/fsspec/implementations/reference.py#L87) for [reading references from parquet](https://github.com/zarr-developers/VirtualiZarr/blob/refactor/backends/virtualizarr/parsers/kerchunk.py#L47-L59) with a fully independent obstore implementation so that our only fsspec dependency would be for the...
Thanks, @sharkinsspatial - that's annoying about Lithops but it definitely makes sense to push that problem to another PR. Would be good to open an issue so we don't forget...
From @sharkinsspatial : > This is a case where as [@Max](https://earthmover-community.slack.com/team/U07SH2MFK5F) outlined in https://github.com/zarr-developers/VirtualiZarr/issues/553 we are caught in a transition period where the open_virtual_dataset signature no longer aligns with the...