opendrift
opendrift copied to clipboard
new version of functioning xarray IO interactions
I have integrated all the changes and minor bug corrections I observed I can (in my setup) read and write CF and UGRID compliants files The only thing that I had to modify that may have an impact is a line in base_model.py that wants to import the file before you write it really. It is meant as a source of inspiration more than an all rounder. It considerably accelerates my usage by reading directly zarr through xarray and writing trajectory files in xarray with daskarray buffers.
Not sure why it cannot be installed, perhaps some cache from earlier. My own PRs yesterday passed through fine. Perhaps you can make a new, clean PR, starting from latest on master?
Also it is not toally clear what this brings, can you make an example script for the gallery demonstrating usage? And btw, it is possible to open Zarr datasets with Xarray and provide to reader_netCDF_CF_generic. So it should not be necessary with dedicated readers for Zarr specifically.
I removed the specific zarr reader as it was an old file. I don't think merging into the netcdf reader is a good idea as it is not using xarray The reader specific to xarray uses xarray methods that are not exactly the same as netcdf. You can open a netcdf with it if you want. There is a new attribute called "engine" it is a term coined by xarray (not me) But the xarray reader reads zarr which is 3 times smaller, bomb proof and can be read and written in a mulithreaded way which is near impossible with netcdf. You can also write with IO_xarray which was not actually working in the former versions. Finally it uses dask under the hood which has multiple optimisations. I have been using the code for 2 years and I think I have saved months of computational time using it Still not passing the tests but they are different errors Will look into that later