scream icon indicating copy to clipboard operation
scream copied to clipboard

Allow I/O to process only a subset of a field

Open bartgol opened this issue 2 years ago • 0 comments

There are some fields computed by rad that are only relevant in certain areas (e.g., on cols where it's day), and that are set to something like "fillValue" elsewhere. When doing averages in I/O, it would be best to only tally "valid" entries, so that we do not pollute averages with invalid values.

Possible path:

  1. make output class check if a field called "io_mask" is present in the Field's extra data.
  2. if io_mask is present, we must
    • for avg output: create a counter field, which will be used as denominator for avg, to divide the tally only by the number of "valid" snapshots for each array entry
    • in the run method, call a different combine method (or add an optional arg, the mask), which only updates the tally if the mask entry is ON

Notes:

  • the mask tally must be also added to the rhist file
  • need to handle 0/0 case in avg. Perhaps set avg=fillValue in that case?
  • we should require shape(io_mask)==shape(field). Even if the mask is conceptually only varying horizontally, same shape makes the feature more general, as well as much simpler to implement.

@brhillman feel free to add more context and/or clarify/correct.

bartgol avatar Jun 22 '23 18:06 bartgol