HAP SVM processing should reject visits that have guide star failures

[stscijgbot-hstdp]

Issue HLA-1146 was created on JIRA by Rick White:

Currently the HAP SVM processing (and possibly the MVM processing too) generates products from visits that have known guide star/pointing failures, as indicated by the GSFAIL flag in the database. That leads to the creation of bad images and catalogs.

The HAP processing should reject visits that include GS failures. Neither images nor catalogs should be generated for those visits.

There are a couple of issues that will make this non-trivial to implement:

The QUALITY keyword in the ipppssoot_flt/flc.fits image header does not necessarily include the GSFAIL flag. In at least some cases it is necessary to look at the association fits file ipppss010_drz/drc.fits to find a header keyword with the correct flag value. Alternatively the flag information can be retrieved from the data_quality table in the DADS database.

After this change is implemented, it may be necessary to remove some previously generated HAP products from the CAOM database. Possibly that will happen automatically if CAOM is requested to ingest products from an empty visit.

More information with an example is given in the comments below.

[stscijgbot-hstdp]

Comment by Rick White on JIRA:

An example visit with a problem is hst_16657_1x_wfc3_uvis_f275w_iep31x. This is a bad image that shows signs of severe blurring from pointing errors during the exposure. Here is a link to the interactive display for that image, where the problem is apparent.

The archive information page for that visit indicates that it has bad guiding:

Data Quality: GSFAIL; TDF-DOWN
Quality Comment: Guide star acquisition failed. Observation taken on gyros only. Observation completed with gyro guiding.

There are two exposures in this visit, iep31xugq and iep31xuhq. The flt and flc files for those exposures have blank values for the QUALITY keyword in the primary header. However, the association data products (iep31x010_drz.fits and iep31x010_drc.fits) to have a QUALITY keyword that indicates the problem:

QUALITY = 'GSFAIL; TDF-DOWN'

So extracting this information is not as easy as checking the QUALITY keyword in the flt/flc files that are being used for the HAP processing. If the association drz/drc files are not available, the keyword is also available in the DADS databases. For example, this query runs on the replicated database on pwharpo (the same results are available from table pdq_summary):

select pdq_data_set_name, pdq_quality,
    concat_ws(' ', pdq_comment_1, pdq_comment_2, pdq_comment_3) as pdq_comment
from dadsops_rep.dbo.data_quality
where pdq_data_set_name like 'iep31x%'

Here is the output:

|| pdq_data_set_name || pdq_quality || pdq_comment || | IEP31X010 | GSFAIL; TDF-DOWN | Guide star acquisition failed. Observation taken on gyros only. Observation completed with gyro guiding. |

[stscijgbot-hstdp]

Comment by Rick White on JIRA:

Lisa Sherbert pointed out on Slack that ticket HSTSDP-2001 is relevant to the issue that that the QUALITY keywords are blank for the flt/flc files. The EXPFLAG keyword in the headers does not explicitly indicate guide star failure but does indicate that the take-data flag (TDF) is down. E.g., here is the value of EXPFLAG for iep31xugq_flt.fits:

EXPFLAG = 'TDF-DOWN AT EXPSTART' / Exposure interruption indicator

That might be a useful work-around for the missing QUALITY information.

[stscijgbot-hstdp]

Comment by Rick White on JIRA:

In case the EXPFLAG keyword turns out to provide a solution for this problem, I did some more investigation of its values. Here is a SQL query (that runs on pwharpo) to the the counts of all the values it takes on in the database for ACS, WFC3 and WFPC2:

select count(*) as n, sci_instrume, sci_expflag
from dadsops_rep.dbo.science
where sci_instrume in ('WFC3','ACS','WFPC2') 
group by sci_instrume, sci_expflag
order by sci_instrume, count(*) desc

And the results: ||n||sci_instrume||sci_expflag|| |96780|ACS|NORMAL| |16290|ACS|UNKNOWN| |911|ACS|EXCESSIVE DOWNTIME| |196|ACS|MULTIPLE| |36|ACS|INTERRUPTED| |2|ACS|NULL| |123844|WFC3|NORMAL| |44816|WFC3|INDETERMINATE| |23555|WFC3|TDF-DOWN AT EXPSTART| |8002|WFC3|MIXED| |898|WFC3|MULTIPLE| |16|WFC3|NULL| |14|WFC3|INTERRUPTED| |184565|WFPC2|NORMAL| |1915|WFPC2|INTERRUPTED|

There are differences between the instruments. It looks like NORMAL is common across the instruments and is the most frequent value. I do not know whether some of the other values (UNKNOWN, MIXED, MULTIPLE) indicate failures or not.

Note that WFC3 in particular has a high fraction of apparent failures, with only about 60% of the WFC3 datasets marked as NORMAL (compared with 85% for ACS and 99% for WFPC2). So it is probably necessary to understand which flags are truly bad to accurately reject the guiding errors.

[stscijgbot-hstdp]

Comment by Rick White on JIRA:

I spent some time working on this for the current HAP data and wrote up my results as an innerspace page. My recommendation is that for WFC3 we should treat any flag in the pdq_summary database as a reason to reject the HAP products. For ACS, there are combinations of the flags and comments that do not appear to degrade the data, so a more subtle approach is required.

I have not looked at the WFPC2 data products yet to see what approach they will require.