xdem icon indicating copy to clipboard operation
xdem copied to clipboard

Dask for XDEM

Open ameliefroessl opened this issue 9 months ago • 0 comments

Use dask to improve the memory usage of the BiasCorr methods.

Remaining todo item:

  • [x] Map remaining logic from _postprocess_coreg_apply_rst() to _postprocess_coreg_apply_xarray
  • [x] bias_vars mask that you mentioned https://github.com/GlacioHack/xdem/pull/525#discussion_r1608774867
  • [ ] Adapt other methods
    • [ ] Method ...
  • [x] Generic method for map_blocks in the fit function
  • [x] Create unit tests for the introduced functions
  • [x] Fix unit tests
    • [x] Update the examples files to use the new COG files
    • [x] Create a mask file on the fly vs adding a new file to the example data?
    • [x] Once #531 merged to upstream, rebase
  • [x] Save output to file that works out of memory
  • [x] Update geoutils dependency to 0.1.6
  • [x] Rebase to xDEM main
  • [ ] Add documentation on how to load the inputs such that the data is processed with dask. (rioxarray + chunks)

ameliefroessl avatar May 21 '24 15:05 ameliefroessl