sunpy icon indicating copy to clipboard operation
sunpy copied to clipboard

Workout how to get the Quality information from GOES FITS

Open nabobalis opened this issue 2 years ago • 1 comments

WIth #6290 merged in, we have quality flags for the netcdf files but not the FITS.

We need to unify that.

See the comment below:

When looking at the FITS header the following information is contained, e.g. for https://umbra.nascom.nasa.gov/goes/fits/2019/go1520191210.fits:

import sunpy.io
from sunpy.util.metadata import MetaDict
from collections import OrderedDict

filepath= "./go1520191210.fits"
hdulist = sunpy.io.read_file(filepath)
header = MetaDict(OrderedDict(hdulist[3].header))
print(header)
print(hdulist[3].data)
>FITS_rec([(0.55999994, [0., 0.])],
         dtype=(numpy.record, [('Time', '>f4'), ('Status', '>f4', (2,))]))

The following doc is provided for Status, however it is not clear to me how the status maps to xrsa and xrsb (e.g. in netcdf format there is a value for each channel and measurement):

Convert status words to long integer then examine bits.Status 1 - octal mask, description (incomplete list):

  • 1000, Sun eclipsed by MoonStatus 2 - octal mask, description (incomplete list):
  • 1, X-Ray detector off
  • 2, X-Ray detector being calibrated
  • 4, X-Ray Transient
  • 10, X-Ray short channel saturation
  • 20, X-Ray long channel range change
  • 40, X-Ray short channel range change
  • 200, X-Ray long channel saturation')

Does someone know how to extract this information?

Originally posted by @mariusgiger in https://github.com/sunpy/sunpy/pull/6260#discussion_r900046426

nabobalis avatar Jun 21 '22 22:06 nabobalis

I have spent some time comparing corresponding FITS and netCDF GOES files to see if I can find the quality information. GOES netCDF files (hosted at NOAA; where Fido searches) have the quality flags for each channel/measurement, as mentioned above, stored as variable data. I have not found any mention of quality flag information in the GOES FITS files hosted at SDAC. The FITS notes page does not provide any indication that flag information was preserved during the creation of the files.

As an example, the corresponding netCDF file for the FITS file linked to above indicates that there should be quality flags for 87 xrsa channel measurements (all corresponding to Spikes). These do not appear to be in any extension of the corresponding FITS file. I have not found any notes mentioning if the status information is related to quality flags. If anyone knows of other information or resources I could check, that would be helpful.

mwhv2 avatar Sep 08 '22 18:09 mwhv2