Tom Nicholas

Results 1090 comments of Tom Nicholas

> is it generally possible to get the dmrpp from an opendap source? I have no idea, but one of @ayushnag @betolink @Mikejmnez will know.

Looking at the implementation of `xr.open_mfdataset`, I'm becoming optimistic that we can parallelize reference generation for concatenation along any number of dimensions just using `dask.delayed` or `lithops.map`. Basically we just...

An implementation of `open_mfdataset` was added in #349, but I won't close this yet because there are still some todos: - [x] Test that the lithops / dask parallelization actually...

With #590 I think this can be closed. There are a couple of small known bugs but the function exists, is documented, and works.

It's very of-its-time, and all architectural information should also be present in the public docs (especially the page on data structures), so let's just close.

Having a public roadmap would be nice (see also https://github.com/zarr-developers/VirtualiZarr/issues/451) but the roadmap file here is more like a nitty-gritty development checklist for a specific moment.

Hmm, this is potentially an issue with the whole "readers as creators of ManifestStores" idea. We can put this inlined data into a virtual dataset and into Icechunk, it just...

BTW a similar thing would happen if you try to load kerchunk references that are inlined into the kerchunk reference file. But in that case it's easier to generate a...

> Can you try removing that normalization from the virtualizarr side? Yeah we should just remove these checks. That would solve #582 also. @norlandrhagen would you have time to have...

:+1: > Maybe once it's a bit more stable we can switch out mypy. We'll wait for an official release announcement from astral before trying it.