Weird color tint for dark images

[IjhUDWzf]

Describe the bug

For images shot in very dark environment, when I increase the exposure during the processing (e.g. by "compensate for camera exposure"), there is an unpleasant pink/purple color tint. Interestingly, the preview of the image does not suffer from this issue and the final exported image is somewhere in-between the preview and the main view. Please see the screenshot with all three. The raw file as well as the xmp file and exported file are included. Because of the limitation on the uploaded file, the single zip is split into multiple ones and have changed extensions, please use "mv example.z01.zip example.z01 && mv example.z02.zip example.z02 && zip -s0 example.zip --out existing.zip" to recreate a regular zip file. example.z01.zip example.z02.zip example.zip

Steps to reproduce

1. import the provided RAW image
2. (optional) activate denoise profiled
3. export image
4. compare preview, darkroom view and exported image

Expected behavior

The overall color of the exported image, the main view and the preview should match. In my case, the colors from the preview are the best.

Logfile | Screenshot | Screencast

No response

Commit

No response

Where did you obtain darktable from?

downloaded from www.darktable.org

darktable version

4.8.0

What OS are you using?

Linux

What is the version of your OS?

tested on Debian 11 (dt 4.0, package manager) and Debian 12 (dt 4.8, appimage)

Describe your system?

No response

Are you using OpenCL GPU in darktable?

None

If yes, what is the GPU card and driver?

No response

Please provide additional context if applicable. You can attach files too, but might need to rename to .txt or .zip

• The issue is in both 4.0 and 4.8 versions.
• RAW
• yes, it does effect fresh edit
• yes, it was tested with clean instance of darktable
• it is not affected by Lua

[ralfbrown]

That purplish (actually magenta) cast in dark areas when pushing exposure or at very high ISOs is due to the white balancing (which increases red and blue compared to green) and an imperfect black level - there is some variation from camera to camera within a model, and my experience is that the sensor's dark current increases as it ages, increasing the effective black level.

You're also seeing effects of downsampling and noise reduction smoothing pixel values, and probably different color profiles for export, all of which can lessen the apparent strength of the effect.

The workaround is to slightly bump up the black levels in the raw black/white point module.

[IjhUDWzf]

@ralfbrown Thank you for your answer. I tried to set the black levels as you suggested. While I got a little better result in the darkroom, the difference of the exported file to the darkroom is even bigger and the final result is probably even worse. The preview is now greenish. Moreover, the overall color of the original preview is still the best by far. (Since this is the first time I was setting the raw black/white point, there might easily be a better tweak than I did.)

Although I understand that every camera differs and that is becomes more prominent for this type of image where noise is more prominent, it seems to me that the three images (darkroom, preview, export) should be consistent and since the preview looks correct it should not be impossible to do something about the issue of magenta cast.

[ralfbrown]

If it's now greenish, you over-corrected. It takes only a few counts, e.g. from 600 to 605....

[jenshannoschwalm]

Not sure if this isn't a simple white-balance problem. Wouldn't be the first time a camera doesn't get that right with "few coloring" After auto-mode in temperature and disabling color calibration looks much better

[IjhUDWzf]

@ralfbrown

If it's now greenish, you over-corrected. It takes only a few counts, e.g. from 600 to 605....

What I meant is that there always seems to be a difference between the darkroom view and the preview, the better color in one, the worse the other. Moreover, I was not able to remove the tint completely.

@jenshannoschwalm

After auto-mode in temperature and disabling color calibration looks much better

You are right that it looks better, yet at least in my case, there is still slight difference between the main view and the preview while the tint still creeps in from the bottom right.

One more thing that came to my mind...

...and probably different color profiles for export, ...

I do not have any easily visible difference between the darkroom view and exported image for other shots.

[jenshannoschwalm]

Just the facts, we have difficult white balance plus a large amount of (high frequency) chroma noise in this image.

The difference in visual output between preview/main-canvas in dt is well explained by the involved downscaling.

1. the preview pipe data are downscaled before anything else for the sake of best speed.
2. for export or main canvas the downscaling happens much later in the pipe

Any downscaling might by very tricky, there is a lot of research papers available.

1. "Content aware" scaling has not been implemented yet for dt, also all algorithms doing so are more likely for errors with a lot of noise
2. If data frequency is close to nyquist we get even more problems. We might need to add some anti-aliasing filter. I am not yet sure this is a problem here but it looks suspicious. (You might be interested in #13335, look at the world-light image how scaling changes tint)

[github-actions[bot]]

This issue has been marked as stale due to inactivity for the last 60 days. It will be automatically closed in 300 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.

[Etzeo]

It's one year in the future, and I'm facing similar issues on my raw files from my sony a6700. It's most apparent in low light / underexposed situations. Below is a screenshot of the similar behavior as previously reported where the preview's on the top left and at the bottom bar looks correct without the magenta cast/tint while the main image and the exported image does. From my observations, it does not appear to be a simple white balance issue.

Just out of curiosity, I also opened the image in sony's Imaging Edge Desktop app, and it does not have this issue, the editor displays an image that's more aligned to the darktable previews on the top left and bottom bar. that can be seen int he 2nd image attached

I'll happily share the raw file if anyone is intrested, it's too large for Github attachments

[wpferguson]

The darktable images in the bottom bar are taken from the embedded jpg in the raw file, so they are not the RAW, they are a processed JPG. When you open your raw file in Sony's software, the software processes it to look like the JPG and that's what you get as a starting point. When you open the RAW in darktable, you get the RAW with the minimum amount of processing necessary to display the image.

[Etzeo]

@wpferguson

The darktable images in the bottom bar are taken from the embedded jpg in the raw file, so they are not the RAW, they are a processed JPG. When you open your raw file in Sony's software, the software processes it to look like the JPG and that's what you get as a starting point. When you open the RAW in darktable, you get the RAW with the minimum amount of processing necessary to display the image.

The next image on the right on the bottom darktable bar is the original JPG that came with this raw. Notice how the colors are a bit different, so I don't think the preview and the bottom image is showing the embedded jpg?

So am I understanding correctly that the bottom bar and the top left image are applying certain adjustments on top of the embedded jpg instead of a downsized version of the raw?

The Magenta tint/cast is just part of the original raw file? I don't expect the image to look identical to the jpg, but the magenta does not seem normal? If this is a common/expected behavior, is there a term for this magenta tint in the shadows so I can research and better understand it? I don't get many results looking up magenta cast/tint.

I struggle to recover the image into a workable state without crushing the shadows, or affecting the rest of the colors?

[wpferguson]

When you import an image into darktable the embedded jpg is used to generate a cache of thumbnails for the lighttable and filmstrip. These cached images are used to represent the image until you open the raw, process it, and save the processing by returning to lighttable or moving to the next/previous image. When you return to lighttable the cached image will be replaced a down sampled copy of the raw.

The magenta tint is most likely chroma noise, since you were shooting at night at ISO 8000. Denoise the image and apply a contrast curve and you'll be much closer to what you are seeing with the jpg. And remember, the jpg you are looking at has been denoised and had a contrast curve applied to it along with any other processing that Sony does in camera.

[Etzeo]

@wpferguson Thanks for the tips, I've found applying the denoise (profiled) module and adjusting the bias correction helps with correcting for the magenta tint. However, I'm not sure what's causing this, perhaps it's how the denoise algorithm works, but here's a few more unintuitive behaviors that I'm noticing, and the last one is a dealbreaker :(

• after editing, the bottom filmstrip and top left preview is still slightly more green tinted (even though it should be updated to a down sampled copy of the raw like u mentioned)
• I noticed simply cropping the image affects the magenta/green tint in the main editor img, but not the previews, they maintain the same tint/bias. Oddly enough, the closer I crop in, the more green tinted gets? And I'll have to adjust the bias correction again.
• lastly, the exported image is very different from what is shown in Darktable editor, which is a giant dealbreaker :(
• (sidenote: The cropping tint issue I mentioned only affects Darktable main editor img's tint. Cropped or not the exported result has the same amount of tint)

Would really appreciate it if you have any wisdom to share on these, especially the last one. Thank you very much!

(attached imgs for reference, I've tried exporting with both high quality resampling on and off)

Noticeably more green tinted in the editor when crop module is enabled, but the exported img is the same (same amount of green shifted from the editor as uncropped, as shown in the img above)

[jenshannoschwalm]

@Etzeo i definitely would be interested in such raw files, either mail me confidentially [email protected] or use a file sharing service

[Etzeo]

@jenshannoschwalm Thank you for taking the time to look into this, I've emailed you with the relevant files, much appreciated!

[jenshannoschwalm]

Well,

1. shooting at ISO 80000 (not 8000) with huge amount of chroma noise.
2. Various light sources with specific lights. So i have no idea what the correct white point might be here as i don't know the scene.

Very difficult to handle. All scaling in darktable darkroom have great difficulties downscaling such data, use HQ processing for better results more comparable with exports. I checked the raw and technically nothing is wrong to me in darktable processing.

[Etzeo]

Thanks for looking into it, I'm still new to taking photos with a camera, and I've just left the ISO on auto, understanding/expecting it will be quite noisy, but wasn't familiar with how intense chroma noise can get. That's for pointing out HQ processing, that resolves the issue with having a different result in the editor vs the exported result.

TLDR for anyone else with this issue in the future (issue i ran into, might be different from OP):

• use denoise (profiled) -> bias correction to compensate for the tint
• toggle on HQ processing on the bottom right if exported result seems different from editor preview

[github-actions[bot]]

This issue has been marked as stale due to inactivity for the last 60 days. It will be automatically closed in 300 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.