SONY ARW, wrong white point messes up the current new Highlight Reconstruction modes

[AxelG-DE]

The error For SONY ARW-files, all new highlight reconstruction modes, like guided laplacian, segmentation based or inpaint opposed show the guided laplacian "Effect", as long the raw whitepoint is not adjusted to what can be found in exiftool (15360 in below example, where dt set 16620)

To Reproduce

1. Go to a darkroom with a SONY ARW raw-file (happily use the sample file from my screencasts attached to this issue)
2. do not adjust the raw black/white point (at current master that is 16620, wheras exiftool says 15360)
3. Click the different options in Highlight reconstructions module, as shown in the screencast
4. See error

Sample pic by me, licensed CC-BY-NC-SA 2018-05-18_141434_DSC03812.ARW.zip

Expected behavior As shown in the "HLR-good" screencast below (after I changed the raw white point to 15360)

Screencast HLR-bad.mp4

HLR-good.mp4

Related other issues To me looks related to #12193

Platform

• darktable version : 4.1.0+474~g9e95d1593
• OS : Linux - kernel 5.19.12-gentoo
• Distro : Gentoo Base System release 2.8
• Processor : Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
• Memory : 32 GB (4 x 8 GB) + 5GB Swap
• Graphics card0 : NVIDIA GeForce GTX 1060 6GB
• Graphics card1 : NVIDIA GeForce RTX 2070 SUPER
• Graphics driver : nvidia-drivers-515.65.01
• OpenCL installed : Yes (opencl-headers-2022.01.04)
• OpenCL activated : Yes
• Xorg : xorg-server-21.1.4
• Desktop : KDE 5
• GTK+ : gtk+-3.24.34-r1
• gcc : x86_64-pc-linux-gnu-11.3.0
• cflags : CMAKE_FLAGS="-march=native-O2-mtune=native-pipe"
• CMAKE_BUILD_TYPE : "Release"

Additional context

• Can you reproduce with another darktable version(s)? yes with version several latest master releases
• Can you reproduce with a RAW or Jpeg or both? ARW only
• Are the steps above reproducible with a fresh edit (i.e. after discarding history)? yes

[gi-man]

This is a duplicate of https://github.com/darktable-org/rawspeed/pull/349 The white point is wrong (too high), so the HR wont try to recover. This needs rawspeed support.

[AxelG-DE]

@gi-man I wouldn't go as far to call it a duplicate but another effect. And note the non-high ISO here.

[jenshannoschwalm]

Sure it's related to the rawspeed issue.

But: there are many images affected not only Sonys. So for darktable it's a somewhat far reaching problem. So for me the title would be somewhat different "how to handle wrong white points in dt" maybe.

And rawspeed is pretty silent BTW.

[Nilvus]

As the issue is on rawspeed, handle it on darktable would just be a workaround (and a bad one when this will be fixed in rawspeed).

So I think this issue should be closed. Said that, that goes for me to the main issue behind that (and some other RAW related issues):

And rawspeed is pretty silent BTW.

I've had feedback four months ago that @LebedevRI is working on rewriting rawspeed. I don't know more since. That goes to a question: extending or not to libraw. Or hoping that rawspeed became again more active. @LebedevRI: if you see that and have a little time to give news about rawspeed and your time/possibilities about it, it would be great.

[jenshannoschwalm]

I didn't want to propose any workarounds. One point would be how to support him or taking data from libraw?

[LebedevRI]

(FWIW, while i've left #darktable channel quite some time ago, was it 2017?, i am present in #pixls.us) The TLDR is that no good deed goes unpunished. I suppose i've become overinvested in everything i was doing, and while that was good while it lasted, in the end nobody really cared, so the end result was that i completely tuned out of all the github notifications, because they were only sending me deeper and deeper down the negative feedback loop. Let me see if i can at least deal with RPU sample queue...

[AxelG-DE]

@LebedevRI Thank you very much Roman! ...for many things, your work, sharing your thoughts, for not being stubborn...

Speeking for me and hopefully for many others, I will be always respectful and not aiming to pull you down, rather support with my little testing capabilities 😊

[LebedevRI]

Let me see if i can at least deal with RPU sample queue...

(Done, ignoring DNG samples... Maybe i will find energy to see look into rawspeed backlog next)

[Nilvus]

A big thank you Roman for taking time to answer and taking care about that. It's completely understandable and taking care of yourself is more important. I hope you will find energy to keep going (without taking too much time and going back to the negative loop) and find someone that could help you for letting rawspeed continue.

[AxelG-DE]

Sorry for boon question... When building master is it enough to init submodules, to get the new rawspeed activated?

@jenshannoschwalm can we close now, or is the HLR reaction (major of this issue) still under investigation?

[LebedevRI]

(Note that i have intentionally not touched issues in this repo, because i didn't propagate the submodule update yet)

[kmilos]

When building master is it enough to init submodules, to get the new rawspeed activated?

Nope, master is tracking the rawspeed stable branch. After syncing the submodules, you need to switch rawspeed to the develop branch.

[todd-prior]

When building master is it enough to init submodules, to get the new rawspeed activated?

Nope, master is tracking the rawspeed stable branch. After syncing the submodules, you need to switch rawspeed to the develop branch.

@kmilos is that easy enough to do?? Sorry I know a little about git but not how rawspeed is integrated into DT . Its no big deal so you can ignore my statement of ignorance on the topic... I was just wondering.... :)

[jenshannoschwalm]

@AxelG-DE I think you can close because of two reasons

1. rawspeed stuff will be merged when ok and a time slot has been found
2. the opposed in combination with clip slider and visual works pretty also for those images in question.

[AxelG-DE]

I still see the effect as described, but agree to your two reasons Closing now