[HOW-TO] modify raw image data before output to preview

[JosephDemarest]

Describe what it is that you want to accomplish Add an effect over the preview output, or preferably, apply some sort of transformation / effect to the image data before passed to the preview.

Describe alternatives you've considered I spent a few hours trying to extend qglpicamera2 with some custom logic, but i hit a roadblock and figured you guys are significantly more knowledgeable on the library than i am. (obviously).

Additional context the cameras will always be capturing in raw mode.

[davidplowman]

Hi, thanks for the question. A few thoughts:

• If you're happy actually to change the image itself, then you can use a pre_callback. For example, have a look what this example does to alter the image by writing text on it.
• If you don't want to change the image itself, maybe you can add something over the top using an overlay. See this example.
• If you want to do something more sophisticated that can't be done with an overlay, then you probably need to edit or extend the preview windows. I'm sure it must be possible to add or change the graphics shaders to accomplish all kinds of different things - but it's not something I really know too much about. All they do at the moment is resample the image, with optional horizontal/vertical mirroring. Maybe there's someone else out there who can help?

[JosephDemarest]

"If you're happy actually to change the image itself, then you can use a pre_callback. For example, have a look what this example does to alter the image by writing text on it."

That is some great advice. Thank you. MY problem is completely solved now. I don't know how i missed pre_callback when post_callback is right there... lol.

I have two more questions for you if you don't mind. I can start another topic if required.

Settings for Pure Raw Image Output: Additionally, I am looking to capture raw images with the camera while ensuring that the output is not modified by any in-built settings like contrast, brightness, AEC, AWB, etc. Essentially, I want to set only the resolution and integration time, and keep the raw data untouched by any other processing or automatic adjustments. Could you advise on which settings or controls I should apply (or disable) to achieve this pure raw output?

[davidplowman]

It sounds like you should be using the "raw" image stream (picam2.capture_array('raw')), which comes directly from the sensor. It is not affected by any of the additional processing, though of course it is affected by the exposure time and analogue gain. But you can set those to whatever you like, e.g.

picam2.set_controls({'ExposureTime': 10000, 'AnalogueGain': 1.0})

By default the values are packed so that you have (for example) 3 pixels in 2 bytes (HQ cam), but you can ask for the unpacked so that you get each value in 2 bytes.

[JosephDemarest]

Thank you. That information is invaluable. Have a fantastic day