spatialdata
spatialdata copied to clipboard
Allow reloading of particular elements, and add `reload: bool` parameter to write functions
With the new incremental IO PR we can write single elements or metadata into a Zarr store. However, reading is still a monolithic operation that reads all at once.
We should
- [ ] refactor the code for reading into individual components to read single elements and metadata of single elements
- [ ] add new APIs to reload the data, metadata and in particular transformations of a single element in a
SpatialDataobject. - [ ] consider adding a parameter
return_stored(as discussed here https://docs.dask.org/en/latest/generated/dask.array.store.html#dask.array.store), or maybereload, to allow the user to callwrite_element()andwrite()and return an object that is already reloaded with Dask.write(reload=True)is equivalent to callingsdata = read_zarr(sdata.path) - [ ]
write_element(reload=True)would introduce a new convenient functionality that has for instance been mentioned by @ArneDefauw here https://github.com/scverse/spatialdata/pull/501#issuecomment-2018315221, and could be used to simplify in particular the workaround 1 for overwriting elements (test_incremental_io_on_disk()).