deeplycloudy
Using 0 as _FillValue is confusing
The processed GLM files use 0 as a _FillValue (at least for flash_extent_density, which is the data variable I'm working with). This is confusing, because 0 is a perfectly valid flash extent density. As a consequence, users cannot tell what pixels have no flashes measured or in what pixels for whatever reason the flash extent density could not be determined.
I imagine this choice may be for visualisation purposes, so that where the flash extent density is zero, the background pixel is drawn (like in the C14_flash_extent_density composite in Satpy), but in that case it should be up to the data visualisation routine to process 0-values accordingly and it shouldn't be masked in the data file.
While I agree that a fill value outside of the valid range of the data would be best, shouldn't you or any tool you are using (xarray) mask out the _FillValue for you?
Yes, the tool (satpy by calling xarray) is masking out the _FillValue. I was wondering why more than 99% of data points were containing NaN, and traced it back to the _FillValue. Now I'm changing them back to zero for the purpose of my statistical analysis, hoping that those pixels actually correspond to zero FED and not to pixels where the FED couldn't be calculated.
Oooohh! So you aren't necessarily wondering why zero versus another value was chosen, but why no FED is considered "invalid/masked" instead of just 0?