Creating Synthetic Exposure/FITS file by adding successive exposures

[gaitskell]

We have been looking in detail at the photometry from our Oculus Pro Camera (Starlight Xpress) with 150 deg FOV. For exposure times greater than 1 sec the light entering the camera from an apparent magnitude 0 star (Arcturus, Vega) will cause the brightest pixel matched to the star to enter a non-linear regime. The non-linear regime starts for a pixel at a level above 70% of max pixel reading. Artifacts such as an oscillation of levels starts to occur between successive pixel exposures when pixels enter the non-linear regime so we want to stay away from it.

Is it possible to create a mode in indi-allsky that sums the raw images from rapid successive camera exposures and simply adds them creating a synthetic exposure. The summed result is saved as the synthetic FITS file. (The process would not save the separate input exposures as FITS files, just the final summed synthetic FITS. The synthetic exposure time would be the sum of the exposure times. The other FITS headers would match those appropriate to the first image in the sequence.)

https://github.com/aaronwmorris/indi-allsky/wiki/Image-order-of-operations

An example of how we would use it: Set a max exposure to 1 sec, and a synthetic image count of say 32. The initial camera process would take 32 images in succession, adding them together to create the synthetic image and synthetic FITS file (This would be in step 2 of the Order of Operations). The exposure time is the sum of the individual exposure times. The synthetic FITS would need to have a larger value range for each pixel. For example, use an unsigned integer 32 rather than original format for camera pixeks (unsigned integer 16) given that the summed image would likely overflow the 16 bits per pixel.

We want to do the summing at the early stage of the order of operations and work with the sythetic summed FITS to reduce the bandwidth required to transfer a large number of separate shorter exposure FITS files.

An alternative approach would be to insert an extra option in the pipeline for the FITS file handling, this occurs after the initial FITS files have been generated. When "n" initial FITS files have been generated a separate process would combined them into a new synthetic FITS file similar to that discussed above. The initial FITS files would be deleted and replaced by the single synthetic (summed) FITS. The synthetic FITS would then be the one that file handling stores, and copies to the remote site as it does currently. This approach has the simplicity of simply adding one new process in the file management queue. It does of course now require a temporary holding directory in which initial FITS files are accumulated prior to summing together.

The goal of summing images from the CCD is to improve the effective dynamic range of the CCD and to avoid the non-linear response regime for each individual exposure.

I realize the above options may not be consistent with your pipeline design. If you can see a way to insert such an option that would be great. Completely understand if it is too complex. I am aware of the existing stack option, but that is only applied to the later enhanced photo images?

As an aside, do you know how accurate the exposure times are? I assume the exposure time is sent to the specific camera hardware, and it determines how long to actually open the CCD wells?