jwst
jwst copied to clipboard
spacetelescope
Enhance jump detection algorithm to work with fewer groups
Issue JP-1885 was created on JIRA by Anton Koekemoer:
This proposed enhancement to the baseline algorithm was received from Mike Regan who has discussed it also with Karl Gordon, who concurs with this.
Currently, the baseline jump detection algorithm only works on datasets that have at least 5 groups (ie, ngroup=5 or larger). However, it is expected that a significant amount of data will be obtained with fewer groups. Therefore, an enhancement is proposed to the baseline algorithm that will enable it to work with datasets that have fewer groups, with the following changes proposed:
For integrations with ngroup=4, do not exclude the highest difference before taking the median.
For integrations with ngroup=3, exclude the larger of the two differences, if it is above some threshold (probably different than the normal threshold).
For multiple integration exposures, process all the differences in the exposure at once. This allows jumps to be found in all integrations with at least 2 groups. This would really help TSO observations.
Comment by Anton Koekemoer on JIRA:
This has been scheduled for discussion at JWST Cal WG meeting 2021-02-02 with a presentation by Michael Regan
Comment by Anton Koekemoer on JIRA:
Linked this to https://jira.stsci.edu/browse/JP-1843 - while these two Jira issues are different, there is sufficient commonality (eg, in terms of the thresholds that would be used for jump detection) to warrant linking them together, even as they are worked separately.
Comment by Howard Bushouse on JIRA:
Handling of exposures with NGROUPS=3 and 4 has been implemented in https://github.com/spacetelescope/jwst/pull/5915.
Reworking the way multiple integrations are handled is still open.
Comment by Alicia Canipe on JIRA:
Anton Koekemoer do you want to create a separate ticket for the multiple integration case, as was mentioned in our quarterly tag-up with the JWSTMO, that way we can close this ticket out when the NGROUPS=3&4 case is done?
Comment by Michael Regan on JIRA:
Alicia Canipe What are you referring to when you say "the TSO case". Is this the multi-integration algorithm?
Comment by Alicia Canipe on JIRA:
Yes, sorry, fixed my comment (it was based on the tag-up w/ the Mission Office in October).
Comment by Anton Koekemoer on JIRA:
thanks Alicia Canipe indeed, as discussed at that meeting the 2-group case is sufficiently distinct that it warrants its own ticket, so I've now created this ticket to help track that work separately JP-2517 and also added the same folks as watchers / assignees
Comment by Anton Koekemoer on JIRA:
This has been scheduled for discussion at JWST Cal WG meeting 2021-02-02 with a presentation by Michael Regan
Comment by Anton Koekemoer on JIRA:
Linked this to https://jira.stsci.edu/browse/JP-1843 - while these two Jira issues are different, there is sufficient commonality (eg, in terms of the thresholds that would be used for jump detection) to warrant linking them together, even as they are worked separately.
Comment by Howard Bushouse on JIRA:
Handling of exposures with NGROUPS=3 and 4 has been implemented in https://github.com/spacetelescope/jwst/pull/5915.
Reworking the way multiple integrations are handled is still open.
Comment by Alicia Canipe on JIRA:
Anton Koekemoer do you want to create a separate ticket for the multiple integration case, as was mentioned in our quarterly tag-up with the JWSTMO, that way we can close this ticket out when the NGROUPS=3&4 case is done?
Comment by Michael Regan on JIRA:
Alicia Canipe What are you referring to when you say "the TSO case". Is this the multi-integration algorithm?
Comment by Alicia Canipe on JIRA:
Yes, sorry, fixed my comment (it was based on the tag-up w/ the Mission Office in October).
Comment by Anton Koekemoer on JIRA:
thanks Alicia Canipe indeed, as discussed at that meeting the 2-group case is sufficiently distinct that it warrants its own ticket, so I've now created this ticket to help track that work separately JP-2517 and also added the same folks as watchers / assignees
Can this ticket be closed? Though, it does not say in which version of the Cal Pipe or build it was included
Comment by Howard Bushouse on JIRA:
The modifications for 3 and 4 groups were included in https://github.com/spacetelescope/jwst/pull/5915, which was in DMS B7.8. The remaining 2 groups case has been separated off into ticket JP-2517, so I would say that this ticket can be closed.
Comment by Misty Cracraft on JIRA:
I have tested this update for MIRI, in which 3 and 4 group ramps are not run through jump detection (no pixels flagged as jumps) due to first and last frames being rejected and too few groups to flag any jumps. The jump step performs as expected for MIRI (not flagging any jumps).