Tom Nicholas
Tom Nicholas
> is it generally possible to get the dmrpp from an opendap source? I have no idea, but one of @ayushnag @betolink @Mikejmnez will know.
Looking at the implementation of `xr.open_mfdataset`, I'm becoming optimistic that we can parallelize reference generation for concatenation along any number of dimensions just using `dask.delayed` or `lithops.map`. Basically we just...
An implementation of `open_mfdataset` was added in #349, but I won't close this yet because there are still some todos: - [x] Test that the lithops / dask parallelization actually...
With #590 I think this can be closed. There are a couple of small known bugs but the function exists, is documented, and works.
It's very of-its-time, and all architectural information should also be present in the public docs (especially the page on data structures), so let's just close.
Having a public roadmap would be nice (see also https://github.com/zarr-developers/VirtualiZarr/issues/451) but the roadmap file here is more like a nitty-gritty development checklist for a specific moment.
Hmm, this is potentially an issue with the whole "readers as creators of ManifestStores" idea. We can put this inlined data into a virtual dataset and into Icechunk, it just...
BTW a similar thing would happen if you try to load kerchunk references that are inlined into the kerchunk reference file. But in that case it's easier to generate a...
> Can you try removing that normalization from the virtualizarr side? Yeah we should just remove these checks. That would solve #582 also. @norlandrhagen would you have time to have...
:+1: > Maybe once it's a bit more stable we can switch out mypy. We'll wait for an official release announcement from astral before trying it.