Update the config files for aligning HST images

[stscijgbot-hstdp]

trafficstars

Issue HLA-1233 was created on JIRA by Jennifer Mack:

The instrument teams request changes to the alignment software configuration files, including both the  fitgeometry and the minimum number of matches required for a successful fit.

https://tweakwcs.readthedocs.io/en/latest/source/imalign.html

1.) fitgeom:  change from 'rscale' to 'rshift'

The plate scale of WFC3, ACS, and WFPC2 images is well-calibrated using the current distortion solutions.  Changes on orbital timescales (due to breathing) and on longer timescales (due to changes in focus) are tiny, with errors <<0.1 pixel.  Furthermore, scale terms due to velocity aberration are well modeled and corrected for in the CD Matrix when 'updatwcs' applies the VAFACTOR during pipeline processing.

Simplifying the fit geometry when aligning to catalogs could significantly improve the quality of the alignment, especially for datasets with a relatively small number of matches.  (Rick White confirms that the HLA pipeline did allow for scale terms... just a simple shift+rotation)

2.) {}minobj{}{}:  increase from 6 to 10{} 

Currently 6 matches is required for an 'rscale' solution and 4 matches for a 'shift' solution.  We would like to remove the shift-only option and increase the required number of matches to 10 and use the 'rshift' geometry instead (see #⁠1) 

 

From Michele De La Pena in team_burgundy slack channel:

This is straightforward to implement as it is controlled by a configuration file.  However, there are some checks in the code, so this will also need to be examined.

[stscijgbot-hstdp]

Comment by Steve Goldman on JIRA:

Jennifer Mack Please let me know if you have a good dataset(s) to test this ticket on. 

[stscijgbot-hstdp]

Comment by Steve Goldman on JIRA:

I was able to confirm that code change results in the desired effect using the program: jczf55 which has 6 nmatches for one of the files (hst_14212_55_acs_wfc_f606w_jczf55iy_flc.fits).

[stscijgbot-hstdp]

Comment by Jennifer Mack on JIRA:

Cool. I was still trying to find an example...  but looks like you found one.

[stscijgbot-hstdp]

Comment by Jennifer Mack on JIRA:

We need some distortion experts to weigh on the estimated accuracy and stability of the distortion solutions. 

Specifically, is there any evidence that fitting a scale term (in addition to a shift and rotation) is desirable for either instrument, and if so, how many Gaia matches would be needed to ensure an accurate fit, e.g. > 50 ?  

Tagging individuals here. We need your input before this change is implemented in the next software update in 1-2 months.

Tri Astraatmadja Roberto Avila  -ACS

Varun Bajaj  - WFC3

[stscijgbot-hstdp]

Comment by Jennifer Mack on JIRA:

Steve Goldman

I wanted to verify that these changes in the fitting logic will apply to:

• all of the alignment code, not just HAP products.
• both RELATIVE (HST-to-HST) and ABSOLUTE alignment (HST-to-Catalog).

 

[stscijgbot-hstdp]

Comment by Tri Astraatmadja on JIRA:

With six parameters, minobj = 10 is adequate if ordinary linear regression is used and there are no outliers in the data set. However, if weighting of individual sources is applied to make the regression more robust against outliers (or if using sigma clipping), it is advisable to increase it. Perhaps twice the number of parameters to make sure if there is adequate number of stars after some are downweighted or rejected… so minobj = 12 if rscale is used?

[stscijgbot-hstdp]

Comment by Jennifer Mack on JIRA:

After more discussion with the instrument teams, we are discovering that the distortion reference file (IDCTAB) is not calibrated as well as we would like and there are still residual scale terms remaining.

Thus, we would like to make a slight change to this request, and we would like this to be part of the next software build in ~June/July.

1.) Keep the original fitgeometry='rscale'

2.) Use the new mimimum number of matches=10.   EDIT ** TBD: We may want to use rshift for nmatches between 10-30 **

3.) NEW  Check whether the code has any additional logic for rejecting a catalog fit, e.g. based on the fitting RMS value.  Michele De La Pena  had mentioned fit RMS values >20mas were a GaiaeDR3 fit.  

---

For a separate ticket (TBD):

We would like to add new drizzlepac header keywords ;  nmatches_rel and {}fit_rms{}_rel* to capture the accuracy of the relative alignment for FIT-REL-* WCS solutions.

the current values for nmatches and fit_rms for the second alignment step, e.g. to the reference catalog.  This will help us better understand whether users may need to reset the WCS to the a priori solution, preserve the fidelity of the sub-pixel dither pattern.

 

[stscijgbot-hstdp]

Comment by Jennifer Mack on JIRA:

We are have been testing the fit residuals for UVIS images with < 50 Gaia matches.

We are still considering using rshift for nmatches between 10-30 and will update this ticket soon!

[stscijgbot-hstdp]

Comment by Jennifer Mack on JIRA:

UPDATE:

We would like to update the logic of the alignment software so that there are {}separate config file settings for relative versus absolute (catalog) alignment{}.  This is important in order to preserve relative alignment of exposures acquired with sub-pixel dithers which optimally sample the PSF.  

For standard HST ipppssoot and HAP hst_ipppss data products, the FIT-REL-catalog solution does a relative alignment between exposures in an ASN and then a catalog fit.  In most cases, the pointing of HST is very accurate and the relative alignment is not necessary. When there are few matched sources, this step can introduce errors and degrade the quality of the combined drizzled products significantly.

PROPOSAL:

 1.) For the {}relative alignment (HST to HST){}, we would like to use different config file settings, e.g. {}Nmatches>=20 and fitgeometry='rscale'{}.  If there are not enough matches, the relative alignment is skipped, the (proposed) new keywords nmatches_rel and fit_rms_rel* are set to N/A and the catalog alignment is performed for the drizzled combined product instead.   In this case, the WCSNAME will still be FIT-REL-catalog, but the additional keywords will let us know what was done by the software.

2.) Keep the current config file settings for {}catalog alignment, e.g.  Nmatches>=10 and fitgeometry='rscale{}', and please confirm whether an upper limit on the fit RMS is folded in somewhere as an additional criteria.

I will create a new Jira ticket for this proposed work.

[stscijgbot-hstdp]

Comment by Michele De La Pena on JIRA:

Steve Goldman We should discuss the request made by Jen on 14 June 2024 to decide the best way to implement such a change and to ensure the modification fits into an overall methodology.  Depending upon the status of this ticket, the update may need to wait to be incorporated into another (post-HSTDP-2024.2.0) build.

If the previous changes have been implemented, I would suggest making the latest request a new ticket.

[stscijgbot-hstdp]

Comment by Steve Goldman on JIRA:

Ticket closed and continued in HLA-1271.

After testing alternative changes have been requested. 

[stscijgbot-hstdp]

Comment by Jennifer Mack on JIRA:

Note to Steve Goldman : 

For the upcoming build, please undo one of the changes in this ticket by resetting fitgeom='rscale' and keeping nmatches (minobj) >=10 for everything. 

(HLA-1271 is development work for a future build.)

Thanks!!!

[stscijgbot-hstdp]

Comment by Steve Goldman on JIRA:

Jennifer Mack Just to be sure, these are changes that you want the images to undergo while aligning to create SVM products, not pipeline products or MVMs, correct?

[stscijgbot-hstdp]

Comment by Jennifer Mack on JIRA:

Hi Steve,

These changes are requested for alignment code for BOTH standard pipeline products and HAP-SVM products.

 (The HAP-MVMs do not get realigned, so does not apply to them.)

 

[stscijgbot-hstdp]

Comment by Steve Goldman on JIRA:

closed by #⁠1823