JWST Unset DQ Flags

[TheSkyentist]

I wanted to clear up some confusion about the default DQ bits unset by grizli for JWST processing.

Currently the pipeline will unset the bits of the result of 1+4+6+32768 or b1000000000001011: https://github.com/gbrammer/grizli/blob/0d5c454afa5b28924bc4db9424db619e9b1a59ae/grizli/prep.py#L6477-L6478

As 6 is not a power of 2, this has the effect of unsettling bits in positions 0, 1, 3, and 15 which correspond to flags: DO_NOT_USE, SATURATED, DROPOUT and TELEGRAPH.

I have the following questions:

1. I've found that unsetting the DQ bit in position 0 (DO_NOT_USE) causes issues where pixels flagged only as DO_NOT_USE are NaNs in the data (e.g. the DQ value for these pixels is identically 1). Are there large losses if this behaviour is reversed?
2. It seems counterintuitive to unset bits in position 1 and 3 by summing together the values of 4 and 6 in the bits parameter. Is this a mistake where the desired bits to unset are in position 1 and 2 or perhaps 3 and 3? Is this a mistake or the intended behavior?

[gbrammer]

This should probably just be bits=2+4+32768 and keep the mask for DO_NOT_USE (1). Or more generally, I think all of the JWST bits are "warnings" other than DO_NOT_USE (1). And grizli overrides 1024 and 4096 for masking (snowballs and/or a manual mask) and CRs identified in AstroDrizzle, respectively. To use this, the logic of the function would have to change a bit since bits is currently interpreted to indicated DQ bits to unset.

[TheSkyentist]

Sounds good, I've opened #217 to try and at least fix the DQ bit bug in the short term. Can you specify what you mean by wanting "to use this" for snowballs/CRs? I think you are suggesting potentially implementing a set_dq_bits function or potentially passing a flat to a more general dq_bits function that allows the user to unset/set, is that correct?

[TheSkyentist]

With #217 merged I think this issue is mostly resolved.