Way of adjusting color balance

[netdown]

What are you trying to achieve? Adjust (not tint) one or multiple RGB channels (color balance). It is required for correcting colors of a picture.

Have you searched for similar questions? Yes, spent hours digging. I have found that with libvips it is achievable by giving an array to the linear function, but sharp's only accepts numbers.

Thank you in advance.

[lovell]

Hello, extending the existing linear operation to accept both numbers and per-channel arrays would be a useful addition.

// current API
.linear(0.5, 1) 

// possible future API
.linear([0.1, 0.4, 0.7], 1) 
.linear(0.5, [5, 8, 2])
.linear([0.1, 0.4, 0.7], [5, 8, 2])

Happy to accept a PR if you're able.

[netdown]

Hi,

I hoped that this really basic functionality is already available somehow. Unfortunately, I have no experience in C++ neither I could figure out how the function is called.

Does the function in operations.cc call libvips or what is the VImage instance?

[lovell]

In terms of the libvips C++ API, the call to linear() on a VImage instance is overloaded to accept either double or std::vector for a and b but at the moment sharp uses the former only.

https://github.com/lovell/sharp/blob/d0feb4156c418188ca5195f7bddc4b8d082157ed/src/operations.cc#L292-L300

[netdown]

Then wouldn't changing/removing (idk if it has "or" syntax) the param type checking be enough?

[netdown]

@lovell I have tried to get this working, unfortunately after finally making a successfull build, I get a module did not self register exception. I just overloaded the function with an std::vector param version (also updated js). Is there a chance you can work on this? I can't believe no one has ever needed this feature.

[netdown]

Meanwhile, I roughly figured out how recomb works and how to get it done with it. Please, update the documentation regarding recomb as neither libvips has a detailed description.

At least provide that the current colors can be retained by the following matrix, where rows mean bands and columns are for, well, I'm still not sure, some kind of amount of RGB colors:

[
[1, 0, 0],
[0, 1, 0],
[0, 0, 1]
}

It would be also useful to have a JS shorthand function for this.

[lovell]

Thank you for taking a look at this; PRs to improve the documentation are as welcome as PRs to provide additional features.

[jcupitt]

Hello, given this matrix:

[[a, b, c],
 [d, e, f],
 [g ,h, i]
]

recomb on an RGB image calculates:

r_out = r_in * a + g_in * b + b_in * c
g_out = r_in * d + g_in * e + b_in * f
b_out = r_in * g + g_in * h + b_in * i

With the right matrix you can do any linear colour space transform (sepia, mono, invert, XYZ, bradford, D50 to D65, etc.). With a small amount of maths you can calculate the matrix from a pair of correlated colour temperatures. It works best in linear colourspaces, obviously.

[nemoDreamer]

Wish I'd seen this issue before trying to figure out via trial and error what recomb does... 😢

From my comment on https://github.com/lovell/sharp/issues/2069#issuecomment-1003840406:

Some kind of guide or basic info on what recomb actually does and what each point in the matrix represents would be really helpful... Not even the VImage library has any information and searching for combinations of "image recomb recombination matrix" returns very obscure results or results related to DNA recombination...

Had to finally figure out via trial and error how I could use this to turn my 50% gray into a specific color, thanks in part to @IuriRiquesoDias's answer, and intuiting that it must be the three channels and their respective mixes.

@lovell sadly, I don't know enough about this to contribute to the docs, sorry...

[nemoDreamer]

@jcupitt do you know of any resources that would help spit out those matrices? Like if I apply a "Photo filter > Warming Filter (81)" in Photoshop, and I know what color values that yellow/orange has, how do I get from that to a matrix?

[lovell]

As always, happy to accept a PR that adds more examples or details to https://sharp.pixelplumbing.com/api-operation#recomb via the JSDocs in https://github.com/lovell/sharp/blob/master/lib/operation.js

[jcupitt]

Hi @nemoDreamer,

nip2 has some colour adjustment stuff, you could have a look at how that works:

https://github.com/libvips/nip2/blob/master/share/nip2/start/_convert.def#L256-L334

That's mostly simply going to CIE XYZ colourspace and adjusting the ratios of X, Y and Z. That has the XYZ ratios for common colour temperatures (A, B, D50, etc.) precalculated, or you can compute the XYZ for a CCT. nip2 has code for this here:

https://github.com/libvips/nip2/blob/master/share/nip2/start/_generate.def#L117-L138

You can be fancier and use Bradford cone space rather than XYZ. It should give more pleasing results. The Bradford matrix is there too.

You need to do these calculations in a linear colourspace (ie. no gamma). scRGB is probably the best one. libvips has a fast anyrgb -> scrgb converter that takes account of ICC profiles.

You can "factorize" recomb, ie.:

out = in.recomb(a).recomb(b)

is always equal to:

out = in.recomb(a * b)

Where a and b are 3x3 matrices and * is matrix multiplication. This means you can do any of the adjustment above in only:

adjustment_matrix = srgb2xyz * xyz2brad * d652d50 * brad2xyz * xyz2srgb
out = in.colourspace("scrgb").recomb(adjustment_matrix).colourspace(in.interpretation)

ie. only one recomb, not five.

[nemoDreamer]

You're blowing my fragile little mind right now, @jcupitt ... I'll need some time to process this visually, but thank you so much for the brain-dump!!

Edit: in the end, not as scary as I expected. Makes a lot of sense 😅

[lovell]

https://github.com/lovell/sharp/pull/3303 implements the original request for the use of arrays with the linear operation.

[lovell]

v0.31.0 now available - please see updated linear docs/examples at https://sharp.pixelplumbing.com/api-operation#linear

[Aminelahlou]

Also if you are using rotation matrix with an angle of theta, the result of recomb should be similar to modulate with a parameter of {hue: theta}, Is that correct? @jcupitt

[jcupitt]

@Aminelahlou exactly, recomb will do any affine transform on a colourspace.