nansencenter
Improved mask functionality
Nansat.watermask(), and the mask-functionality of nansat.write_figure() are convenient, and can be used to e.g. mask land in black as:
s.write_figure(fileName='fig.png', bands=4,
mask_array=s.watermask()[1],
mask_lut={2:[0,0,0]})
Nevertheless, I have some suggestions for further improvements:
- it would be useful to use masks from several arrays/bands (e.g. land as black and ice as white)
- it would be useful to mask a range or condition (e.g. ice_concentration > 70)
Desired functionality, in pseudo-code:
s.write_figure(fileName='fig.png', bands=4,
mask={landmask==1: [0, 0, 0], ice_concentration<70: [255, 255, 255]})
'landmask' and 'ice_concentration' could be input NumPy arrays, or better: names of the corresponding Nansat bands. Thus a second suggestion is to make a convenience function to add watermask as a Nansat band:
def add_watermask(self):
self.add_band(array=s.watermask()[1], parameters=
{'name': 'watermask'})
Sorry, but I would strongly oppose both improvements.
Coding pseudocode translation is difficult and bug risky. And for the user to implement the functionality is trivial: wm = s.watermask()[1] wm[ice_concentration<70] = 3 s.write_figure(..., mask_array=wm, mask_lut={2:[0,0,0], 3:[255,255,255]})
Thus there is no need to add watermask as a band.
However I'll use that chance to promote usage of band 'mask'.
Idea behind is that there is an additional band in the Nansat which has several values (or bits) reserved for various conditions. E.G. 0 - out of swath 1 - cloud 2 - land 64 - open water (or other valid pixel) etc
'mask' is added as a band to the Nansat object. 'mask' is used in Mosaic.average or Mosaic.median the following way:
- from all input images to be mosaiced we take the band with values to be averaged and the 'mask'.
- Masked values (e.g. less that 64) are turned into np.nan and nanmean is calculated
- simple maximum of all 'mask' values is calculated so that the following rules apply: {out of swath and cloud} becomes cloud {out of swath and cloud and land} becomes land {out of swath and cloud and land and open water} becomes open water
Example: I have images with e.g. SST from MODIS.
I have a class ModisImageSST(Nansat) with method add_mask() which adds band 'mask' depending on sst_qual, flags, resolution and other bands in the MODIS file. The 'mask' has the bits marked as above.
ModisImageSST.add_mask() cannot be in Nansat because 1. it is MODIS specific; 2. it is task (project) specific; 3. it is not open source
On 01/07/2014 03:38 PM, knutfrode wrote:
Nansat.watermask(), and the mask-functionality of nansat.write_figure() are convenient, and can be used to e.g. mask land in black as:
s.write_figure(fileName='fig.png', bands=4, mask_array=s.watermask()[1], mask_lut={2:[0,0,0]})Nevertheless, I have some suggestions for further improvements:
- it would be useful to use masks from several arrays/bands (e.g. land as black and ice as white)
- it would be useful to mask a range or condition (e.g. ice_concentration > 70)
Desired functionality, in pseudo-code:
s.write_figure(fileName='fig.png', bands=4, mask={landmask==1: [0, 0, 0], ice_concentration<70: [255, 255, 255]})'landmask' and 'ice_concentration' could be input NumPy arrays, or better: names of the corresponding Nansat bands. Thus a second suggestion is to make a convenience function to add watermask as a Nansat band:
def add_watermask(self): self.add_band(array=s.watermask()[1], parameters= {'name': 'watermask'})
Reply to this email directly or view it on GitHub: https://github.com/nansencenter/nansat/issues/61
I agree with mask concept, this would be useful. Then we could easily let write_figure use this, and my task is solved. Would you like to move (generalise) this concept from mosaic.py to nansat.py?
But it should be possible for the user to decide on the order of the masks. E.g. for the case of {out of swath and cloud and land} it would in some cases be better to have cloudmask rather than landmask, otherwise one would get false impression of cloudfree conditions over land, if there are clouds over sea somewhere.
Though a(ny) plotting function should be able to reorder masks by juggling the bits, I suppose?
Btw, GDAL also has a built-in mask functionality. However, this seems not to be too useful for our purpose, as such masks apparently only define valid/invalid pixels of a single band(?). The proposed bit-mask seems like a better option.
Why wouldn't you use GDAL's masks for that? They are of type byte and only specify that 0=no data and everything else is valid data. See http://trac.osgeo.org/gdal/wiki/rfc15_nodatabitmask This fits your current model going from 0 to 64. I don't know in which cases GDAL uses these masks but I can imagine that it makes sense to use their functionality so that it stays compatible, meaning: one can step out to GDAL's API in case nansat doesn't support some specific function.
In the VRT class I found set_subsetMask() which uses GDAL's CreateMaskBand(), but I'm not sure how this connects to everything else. It seems atm no other code uses set_subsetMask.
I was thinking about using the GDAL masks but I'm afraid it is only a no-data-mask. So in fact only two values are usable: 0 and non 0. This concept is used in gdalwarp, when values from pixels masked by no-data are not used in resampling. We have slightly more rich mask usage - in Figure it sets the colours of background (e.g. land and clouds) in Mosaic it masks from averaging and also accumulates mask of clouds and land.
Ok, I think you're right. The GDAL mask would only be usable for your purposes if there is no nodata value and instead an alpha band defined. But my concern is invalid then, as a nodata value will still lead to a GDAL mask (GDALNodataMaskRasterBand), according to rfc15. I thought that using nansat's separate mask would completely sidestep GDAL's nodata masking. But it's rather an addition I would say, right?
The only improvement I can see here is providing some default values for the mask_lut parameter to Nansat.write_figure(). E.g.
0 - out of swath (dark gray 100,100,100)
1 - cloud (white, 255,255,255)
2 - land (gray, 128,128,128)
This is used when only the mask_array parameter is provided.