iris icon indicating copy to clipboard operation
iris copied to clipboard

Enable lazy computation of wind vector rotation

Open tinyendian opened this issue 1 year ago • 0 comments

🚀 Pull Request

Description

This pull request enables lazy wind rotation, where the pevious implementation triggered data loads in the wind component cubes. Wind rotation can thus be computed out-of-core via Dask. If only one of the two wind component cubes has lazy data, the other cube will be converted to lazy data, following the default behaviour of Dask arrays, and avoiding data loads that a user might not expect. The pull request tries to minimise changes to the code.

A few more comments:

  • Coordinate data is not lazy at this point - I can have a look at lazy evaluation if you think it would be worthwhile, e.g., if the same routine should also handle unstructured datasets with 2D coordinate fields
  • An empty_like masked array creation function was only very recently added to Dask, I thus chose the following workaround to avoid having to upgrade Dask version, ut_cube = ut_cube.copy( data=da.ma.masked_array(ut_cube.core_data()) ) In my understanding, the new cube instance of ut_cube should receive all metadata from the old version, and the empty Dask array that is held by the old cube version changes ownership to the new one via reference passing while receiving its mask "upgrade" - I hope that such change of ownership is safe, I can otherwise get Dask to create a new data array.
  • I added a new test to test_rotate_winds.py which checks if "laziness" is indeed preserved, if results are consistent with non-lazy computation, and if the function modifies the input wind component cubes (as a safeguard against data handling mistakes)

Consult Iris pull request check list

tinyendian avatar Sep 18 '22 23:09 tinyendian