Some thoughts on improving the Tone Equalizer UI

[wilecoyote2015]

Hi all! I'd like to exchange / contribute some opinions on the usability of the Tone Equalizer module. I use the module quite often and noticed some issues regarding the Interface and UX that have grown quite annoying for me over time. As functionality and concept of the tone equalizer is awesome, I'd love it to feature streamlined and easy UX. In the following, I describe the issues and ideas for improvement.

Merging masking and advanced tabs

This is basically https://github.com/aurelienpierreeng/ansel/issues/134 When working with photos, I have to fine-tune / adjust the mask for almost every image. The process is quite laborious:

1. switch to the maskng tab
2. turn on mask display because the mask posprocessing range display does not provide something like a histogram.
3. fiddle with the exposure and contrast compensation
4. switch back to the advanced tab and try if it works well. if not, repeat...

Merging the advanced and masking tabs would provide two benefits:

1. Adjustment requires less steps
2. THe histogram in the advanced tab can be used to assess the mask's distribution and directly enables seeing the effects of mask adjustment with regard to the equalizer controls you tune it for.

Replacing mask exposure and contrast compensation by min / max points and maybe add some gamma-like parameter

Basically, exposure and contrast do some linear tranformation of the mask values, right? But as a user, I am most interested in spreading the mask's values from black to white without over- or underexposing. So I'll basically do some kind of histogram equalization. This is difficult using exposure and contrast because both sliders work interdependently with regard to the resulting min and max values.

So, why not introduce white and black relative exposure sliders like in the filmic rgb module? This would be consistend UI with filmic and it enables the user to control the mask's range easily by setting the min and max luminance values. In addition, something like a gamma parameter or a gray value like in rgb levels would allow further adjustment of the mask's value distribution. This would also be consistent with rgb levels UI and intuitive.

Advanced tab: dragging one node should only affect one node

As A user, I want to have control over my adjustments. I want to make fine andjustments in a controlled manner. Hence, I want to change one node and do not want to drag neigboring nodes along. Currently, based on the curve smoothing parameter, neighboring nodes are dragged when dragging one node. Personally, this drives me mad.

See, for example, many parametric equalizer UIs: There, the nodes can be dragged independently. The effect on the equalizer transmission curve is visualized. Here, the nodes are not necessarily on the line due to the spline interpolation, but this is understandable for the user, as they see that the curve is smoothed and dragging the nodes affects.

For some reason, exactly this behavior can be achieved using the simple tab. But then, visual feedback and control using the histogram view is lost. What is the intention of this UX design? I guess there is some benefit of having the magnetic / smoothed interaction with nodes in the advanced tab, but unfortunately, I don't see it. There are not that many nodes and adjusting each one independently in the advanced tab would quick and efficient for achieving the desired result as quickly and with as much control as possible. What do you think? Does the current smoothed behavior provide real benefits for you?

Use some spline interpolation with finite support

This may be a controverse point. As far as I understand the documentation, the curve interpolation is some gaussian and hence has infinite support. In words, if the last node is moved, this will also affect the curve around the first node. According to the documentation, this is by design, following the assumption that the provided smooth modification of gradients is the most important design goal for interpolation.

But is this assumption valid? Personally, I want to do smooth, but local changes in the gradiation. The global smoothing leads to artigacts like oscillations on strong adjustments, regardless of the choice of the corve smoothing parameter. Wouldn't finite-support interpolation like quadratic or cubic B-Splines provide more precise control while still being reasonably smooth?

[EDIT] Update Preview in realtime when dragging nodes in advanced tab

In the simple tab, dragging the sliders updates the preview in realtime. In the advanced tab, the preview is only recomputed after mouse release and not while dragging. This makes adjustment more cumbersome than necessary in my opinion and it is also inconsistent UX regarding simple and advanced tab.

[EDIT] Option to increase equalization range beyond +-2EV

In some Situations, the fixed range of +- 2 EV is quite limiting. An option to increase the range would be very useful.

What do you think about those points? I'd love to hear some opinions, as I am not sure whether other users / devs share some of my struggles with the Tone EQ UI.

[da-phil]

I remember a vivid discussion about exactly this feature, either in the github issues or in the forum at pixls.us.

[da-phil]

Found it here: https://discuss.pixls.us/t/should-the-mask-adjustments-in-tone-equalizer-be-improved/41374 There are also concrete UI suggestions later in the thread. Maybe continue the discussion there?

[wpferguson]

But as a user, I am most interested in spreading the mask's values from black to white without over- or underexposing.

As a user I sometimes want to only affect the lightest or darkest part of of the image so I move and adjust the mask to that section.

Have you watched Boris's videos on tone equalizer? He has some use cases that broadened my horizons with respect to what tone equalizer is capable of.

I don't want to lose features (flexibility and capability) in the interests of making tone equalizer easier to use, especially when ease of use is often in the eyes of the beholder and their workflow.

For my part I find tone equalizer quick and easy to use. Mostly I use presets and the masking isn't an issue unless I didn't do a good job in camera or got a really challenging scene. For the cases where I want to spread the mask over the entire range I have a lua script that uses the auto pickers to accomplish it. It's good for 90+% of the images I use it on, so every once in awhile I have to do it by hand.

[jenshannoschwalm]

Not going into detail - again, this has been explained multiple times here and on pixls - the nodes and the maths behind the simple and advanced UI are identical, they use an identical spline curve.

Also, it does matter how the nodes are positioned and the spline function is calculated as the pixel calculating math depends on it. So it's not about implementing a UI mockup.

BTW this is also noteworthy for the nodes in the color equalizer ...

So any change in the UI must be done with above considered.

[marc-fouquet]

Regarding exposure compensation / contrast compensation

I have not been using Darktable for that long, but my experience is this:

• The two magic wands for mask exposure compensation and mask contrast compensation appear to be working reasonably well (after learning that it is not the goal to fully spread the light bar).
• But I have to click them when initially activating the module, whenever I change the mask, whenever I apply a preset, whenever the exposure of the image changes.
• This is especially bad in combination with styles, because there is no such thing as automatically applying styles that contain the tone equalizer to a large number of images. Each image needs those two clicks (except with custom scripts, as I have learned in this issue, but this can not be expected from normal users).

How about an option to set those two magic wands to "autopilot", so they automatically trigger whenever mask exposure and contrast need to be updated?

In my opinion, merging the advanced and masking tabs would clutter the UI quite a bit. The masking tab contains a lot of complicated options that can overwhelm users who basically just want simple sliders for changing the exposure of shadows/highlights while preserving local contrast (like they exist in most other RAW development software). If the tone equalizer UI is changed, the goal should be to simplify things for new users and reduce friction for existing users.

Is it frequently the case that the magic wands do not work properly, so it is necessary to manually tweak exposure compensation / contrast compensation?

Regarding the mask

Another thought (on a slight detour): There are other masks in darktable that have the (in my opinion) undesirable property of changing the selected pixels when an upstream module changes the image. Examples are parameteric masks and the shadow/midtone/highlight-masks in color balance RGB. Wouldn't it be better to calculate a stable version of the image once (early in the pipeline) and always use this representation for all masking?

Final thought about the UI

Finally I wonder if anyone really uses the "Simple" tab in the tone equalizer. Contrast- and Color-Equalizer don't have a similar tab. The Tone Equalizer module could be streamlined by removing it.

[marc-fouquet]

I looked a bit at the source code and I wonder if this contrast boost and exposure boost is necessary at all.

As far as I understand it, the current process is this:

Step 1: Either get user input for contrast boost and exposure boost or use the automatic estimator. But since the estimator does not know, what will happen in step 3, this involves guesswork. Step 2: Convert the image to Black & White. At this point the contrast boost and exposure boost is applied. Step 3: Blur the B&W representation of the image, which changes exposure and contrast in an unpredictable way.

These are all 32 bit float calculations that should have enough precision to do it the other way round:

Step 1: Convert the image to B&W without applying anything special. Step 2: Blur the B&W representation. Step 3: Scale the resulting mask to the desired dynamic range.

This would involve no guesswork, so the result should be perfect every time. Or am I missing something here?

[jenshannoschwalm]

If you know what you want you would have to go into coding and see how that works out. Currently i don't think there is a big desire from any dev to get into discussing ideas and details that don't seem to fit into implementation without any code to test/review. So feel free to start developing dt and contribute your work to the project :-)

[Donatzsky]

This would involve no guesswork, so the result should be perfect every time. Or am I missing something here?

I haven't considered it very thoroughly, but I suspect that you are indeed missing something. The first question to answer would be if your simplified approach gives the same result. At first glance, it seems to me that it wouldn't. And if the results are not the same, would it be an improvement.

[github-actions[bot]]

This issue has been marked as stale due to inactivity for the last 60 days. It will be automatically closed in 300 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.

[marc-fouquet]

I looked into the tone equalizer multiple times in the last few months. I am not really satisfied myself with the things I came up with so far, as they don't solve the problems that I see, but I decided to put my current version out here to start the discussion again.

To reiterate my main points:

• The tone equalizer requires that the user understands why the adjustment needs a blur mask (contrast preservation) and why this mask should not blur hard edges (prevent inverted gradient artifacts). This places a burden on the user to learn relatively complicated things.
• The mask has to be adjusted for every single image. So applying the tone equalizer to a batch of images using a style is problematic.
• Also changing the exposure of an image - which usually happens earlier in the pipeline than the tone equalizer - affects the tone equalizer and may invalidate user edits. Basically darktable assumes that the user will set the exposure first and work with the tone equalizer after that. But in reality working on the dynamic range is an iterative process.
• Every other RAW development software has "shadows/highlights" sliders that simply work, without all these problems.

Users probably don't want options to be removed to simplify things. A possible solution could be the introduction of a "Shadows & Highlights RGB" module as a simpler alternative, which either might share code with the tone equalizer or just be an RGB version of the old S&H module.

Leaving theoretical discussions and showing what I have

My current changes to the Tone Equalizer are (except for one minor thing) purely UI:

1. Only two tabs. The "advanced" tab was renamed to "curve".

2. The behavior of the histogram was changed. When normalizing the vertical scale of the histogram, the first and last bucket (which contain all points outside the histogram range) are ignored. This way the displayed histogram will not become very small if a large amount of pixels is clipped outside the histogram range (this might be a valid situation if the user works on only on a part of the dynamic range on purpose). Also the left/right clipping indicators were changed from exclamation marks to orange vertical bars.

3. "Mask Histogram Shift" and "Mask Histogram Scale" are new names for "Mask Exposure Compensation" and "Mask Contrast Compensation". As suggested by OP, these are on the "curves" page, so the user can immediately see, how they affect the histogram.

4. New "Curve Tools" collapsible section. This allows selecting a curve preset and scaling the current curve (i.e. multiply all points with 1.1 or 0.9). Those tools are still a bit rough in terms of usability. Also double clicking the new buttons resets unrelated parameters and I wasn't able to find out why.

5. The content of the former "simple" tab has been moved to a "sliders" collapsible section. This is also an advantage, as the user can see in realtime, how slider changes change the curve. Note that there were discussions in the past about removing the sliders at all, but someone mentioned that they were needed to assign them to MIDI controls.

6. The "bar thing" histogram on the masking page has been removed.

7. Options that the user should usually not need have been moved to an "Advanced" collapsible section.

   • The luminance estimator: A user has be be very advanced to care how their color pixels are turned to grayscale.
   • New luminance estimatior "Rec.709 weights": I wondered why there was no estimator that weights green more strongly than red/blue, as human vision does. So I added 0.2126 red + 0.7152 green + 0.0722 * blue as an option. I am aware that this formula is meant for display-referred color spaces, but most of the other existing formulas seem formally incorrect as well.
   • "Mask Quantization" has been moved to advanced as well. I looked a bit at what this does and tbh I don't see the use-case.

The code is here: https://github.com/marc-fouquet/darktable/commits/2025-02_toneequal_preview/

[jenshannoschwalm]

Why not opening a PR marked as so we all can try out in a better way and discuss?

[wpferguson]

The tone equalizer requires that the user understands why the adjustment needs a blur mask

Maybe if the user is trying to do advanced things with tone equalizer, but in everyday use I think not. I think most users click on it to start it and then hover over what they want to adjust and make changes, unless the mask isn't where it needs to be.

The mask has to be adjusted for every single image.

I process thousands of images per month, all with tone equalizer. I find the more than 98% of my images require no adjustment to the mask. Almost all of my images are reasonably well exposed, and maybe that's why. Also, I'm just doing basic targeted adjustments and not "art", so I'm not slewing the mask to only act on a part of the histogram.

Also changing the exposure of an image

I put tone equalizer below exposure in the pixelpipe for this reason. I use tone equalizer for targeted adjustments and exposure for overall. If I use targeted exposure adjustments, they also go below the default exposure module

Shadows & Highlights RGB

Why not a couple of instances of exposure with a parametric mask? I'm not sure we need another module to adjust shadows and highlights.

The behavior of the histogram was changed.

I'm not sure if this is a good idea or not. Expanding the histogram gives more information about transitions, but alters the true relationship between them (i.e. the histogram shows a great difference, but the image does not).

---

I agree with @jenshannoschwalm, submit a PR and let's play with it. You might also ask Boris to play with it since he probably knows more about how to work it than just about anyone else.

[marc-fouquet]

I am working on a better version right now. I will open a PR when I have something that I am happy with.

[europlatus]

@marc-fouquet There has been quite a bit of discussion on pixls.us about this module and possible improvements. A recurring theme seems to be the adjustment of the mask, in terms of needing to click the magic wand every time, and how best to set the mask when working on just one part of the tonal range, e.g. shadows or highlights. The magic wand works well when you want to work on the whole tonal range, but trying to work on just a few EV is much more of a fiddly process.

Have you considered a manual (or possibly auto) way to select the darkest/lightest points from the image that you want to manipulate? A bit like how RGB levels works, where you set the black/white points with colour pickers. So, you could click the blackest black and whitest white to work on the whole tonal range; which would be similar to what the magic wand does now. Or you could equally select a mid-gray as your darkest point and whitest white as your lightest point to just work on the highlights. This tonal range would then be spread over the 9 nodes.

I also think it would be good to get @s7habo's (Boris) input.

[marc-fouquet]

First of all thanks for your input. It is encouraging to see that there are other people who see potential for improvement. Clicking the magic wands every time was exactly the starting point that bothered me as well.

Some disclaimers: I do not have a lot of time for coding AND this is my first time working on a module, so my progress is somewhat slow. Also I don’t like sharing unfinished work (where I can’t prove that my ideas are actually functional). But maybe I should be more transparent, so here is a rough overview of my current plan:

I have returned to the idea of reversing the order of operations that I already mentioned last August. In my opinion this is not just computationally better, but also in terms of semantics (what happens to the image).

In the current tone equalizer, the mask calculation works by first converting the image to grayscale. The result is exposure/contrast boosted and THEN the guided filter is calculated. The problem is that the guided filter changes the dynamic range in an unpredictable way. So when you adjust the contrast/exposure boost sliders, the histogram moves roughly in the direction that you wanted, but with a lot of randomness added on top. The magic wands are some heuristics that try to adjust the mask exposure despite the random nature of the filter.

Have you considered a manual (or possibly auto) way to select the darkest/lightest points from the image that you want to manipulate? A bit like how RGB levels works, where you set the black/white points with colour pickers.

I think this would not solve the core problem, it would just be a different way of getting input from the user. If the guided filter does random stuff with the picked brightnesses, the situation would be worse than before.

I have a basic prototype where the guided filter is calculated first and THEN sliders equivalent to contrast/exposure boost are applied. This feels VERY different, you can move these sliders and the mask actually does what you intended it to do (and these sliders are directly below the histogram, which also helps the feeling of interactivity).

In the next step, I would like to add new automatic alignment options based on this, so the histogram can be automatically fitted to the full tonal range or auto-aligned to the left or right to edit only shadows/highlights. I would make fitting it to the full tonal range the default option. The hope is that this would make fiddling with the mask completely optional, to have a state where the adjustments simply work and the user does not need to care about technical details.

Further ideas:

• I would move the EV-sliders from the simple tab to a collapsible section in the advanced tab, like I did in the image above (5).
• However I want to introduce a new “simple” tab that just contains two sliders for shadows & highlights that do something very robust.
• I also want to store information about the dynamic range of the image before the guided filter calculation. This way the tone equalizer can know, by how much the mask exposure/contrast differs from the image exposure/contrast. The module could show the user what he is doing in image EV (and not mask EV) terms and for example warn the user, if he adjusts the exposure too much (by more than 1EV per 1EV, so darker areas become brighter than originally bright areas, don’t know if this explanation makes sense).

[wpferguson]

Will your changes keep compatibility with existing edits? If not, then this needs to be a new module.

[marc-fouquet]

I know. Don't want to promise too much about not yet written code, but I am aware that this requirement exists.

[europlatus]

I think this would not solve the core problem, it would just be a different way of getting input from the user.

I'm interested to see a working prototype so we can give feedback. I like a lot of what I'm reading but I think it's important that manual input from the user is still an option for those that want it. When you say the core problem, I'm assuming you mean the need to set the magic wand every time. But I think another core problem is the lack of precision in the module. In its current state, it's not clear where the black/white points are, where the mask truly starts and ends, how to dial it in precisely to just the EV you want to work on, etc.

Thanks for your work on this. I'm very interested to see what comes of it!

[todd-prior]

I'm assuming you mean the need to set the magic wand every time.

I haven't used it for a while but in testing the "Initial Workflow " script had a coded option for setting these and while doing it with the pickers sometimes took more than one try to get the basic balanced mask. I feel like running the script set it correctly most if not all times out of the gate...

Might be worth checking the code....

[Donatzsky]

I have been meaning to create a feature request for this, but I figured it might be better to put it here now.

The one thing that rgb curves still has over tone eq is being able to work on a single color channel. While it's possible to choose the affected channel using blend modes in tone eq, the mask is always based on the full RGB value. If I'm understanding how it works correctly, adding a mask input channel option with a choice of [all, red, green, blue] seems like the neatest solution. The output can then, as now, be chosen with the appropriate blend mode.

So when you adjust the contrast/exposure boost sliders, the histogram moves roughly in the direction that you wanted, but with a lot of randomness added on top.

That might explain why it can feel a little "odd" when dialing in the histogram, often requiring more trial and error than expected.

[europlatus]

The one thing that rgb curves still has over tone eq is being able to work on a single color channel.

I would say another thing that RGB Curves has over Tone EQ is the ability to pick an exact point on the curve from the image using a color picker. The closest thing that Tone EQ has is its mouse hover feature, but it's less accurate, resource intensive, and I believe it's based on the mask rather than the image. I really miss RGB Curve's level of precision in Tone EQ.

[wilecoyote2015]

BTW, AP merged the advanced and masking tabs in ansel https://github.com/aurelienpierreeng/ansel/issues/134

[github-actions[bot]]

This issue has been marked as stale due to inactivity for the last 60 days. It will be automatically closed in 300 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.