xESMF icon indicating copy to clipboard operation
xESMF copied to clipboard

Implement ESMpy mask handling

Open jhamman opened this issue 7 years ago • 8 comments

This comes from the discussion in #22. It is intended to help users specify masks for the valid cells in the source and/or destination grids. It does this in the following way:

  • looks for mask variable in ds_in and ds_out
  • if mask variable is present, it adds that mask to the ESMF grid object
  • specifies the mask values in the ESMF.Regrid() call.

I've been testing this and it seems to be headed in the right direction. There are certainly additional features/tests/docs that this will need but I'm putting up here to aid in our discussion.

jhamman avatar Jun 22 '18 19:06 jhamman

Codecov Report

Merging #23 into master will decrease coverage by 0.21%. The diff coverage is 91.66%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #23      +/-   ##
==========================================
- Coverage   95.19%   94.97%   -0.22%     
==========================================
  Files           6        6              
  Lines         229      239      +10     
==========================================
+ Hits          218      227       +9     
- Misses         11       12       +1
Impacted Files Coverage Δ
xesmf/frontend.py 93.75% <100%> (+0.23%) :arrow_up:
xesmf/backend.py 94.8% <85.71%> (-0.97%) :arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 97ec30a...bf712c0. Read the comment docs.

codecov-io avatar Jun 22 '18 19:06 codecov-io

Here's the same example I showed in #22, but using a mask.

image

I haven't figured out yet how to mask the grid cells with nans where there were no valid weights. Perhaps @bekozi knows where to get that info from the ESMpy objects?

jhamman avatar Jun 22 '18 19:06 jhamman

I haven't figured out yet how to mask the grid cells with nans where there were no valid weights. Perhaps @bekozi knows where to get that info from the ESMpy objects?

We could potentially pull in the unmapped destination list to ESMPy, but I think it is easier to use fill values (or nans) during the sparse matrix multiplication. Basically, fill the destination matrix with nans before applying the sparse matrix. Afterwards, if the masks are set up correctly, only the mapped values will be non-nan. See filling and finding for mildly relevant examples.

bekozi avatar Jun 22 '18 20:06 bekozi

Thanks for the PR (the first PR for xESMF)!

I still have some concerns: https://github.com/JiaweiZhuang/xESMF/issues/22#issuecomment-400175414. Let's continue the discussion in the main issue.

JiaweiZhuang avatar Jun 26 '18 04:06 JiaweiZhuang

@jhamman - this PR was discussed today at the xESMF dev meeting. There was a consensus that it's a very important feature.

Could you let us know whether you plan to keep working on the PR? Do you need any help from the dev team? If you don't have time to finish it, let us know so someone else can take it over.

rabernat avatar Jun 15 '20 19:06 rabernat

Thanks for the ping. I'm not likely to have time to pick this up in the near future so would welcome someone else stepping in if there is time/interest.

jhamman avatar Jun 16 '20 03:06 jhamman

What is the difference between this and #82? Does that supersede this?

If we want to move forward with this PR, I recommend it be moved to https://github.com/pangeo-data/xESMF? Unfortunately this has to be done manually.

rabernat avatar Jul 24 '20 13:07 rabernat

Comparing across forks:

JiaweiZhuang:masking is up to date with all commits from jhamman:feature/masks.

so this PR can be closed

raphaeldussin avatar Jul 24 '20 22:07 raphaeldussin