jwql icon indicating copy to clipboard operation
jwql copied to clipboard

Avoid calling ramp-fitting or jump step on large files

Open bhilbert4 opened this issue 3 years ago • 0 comments

We recently ran into a case where the dark monitor was working on a MIRI file containing hundreds or thousands of groups. It took 2 full weeks for the pipeline to complete. It seems like the return on something like this is not worth the wait. We should insert some logic somewhere such that no monitor calls the jump and ramp-fitting steps of the pipeline are not called on data files this large.

At the moment this would include the dark monitor, and maybe the bad pixel monitor. I don't think the readnoise or bias monitors call these steps, so they probably don't need any changes.

bhilbert4 avatar Sep 29 '22 18:09 bhilbert4