spectral.js
spectral.js copied to clipboard
rvanwijnen
why having the RGB spectrum while its spectrum has a high correlation with the CMYW?
Hi, I plotted your cmy and rgb spectrums and they see to have a high correlation then why having them both?is there any specific case that combining these two instead of just cmyw results in a more realist color? also, these two spectrums look perfect and there is no noise, do you think if you introduce noise to these spectrums results will get more realist?
@donyakh There are indeed cases where you will get worse curves if all are not included.
For instance, when I implemented this in my own library, I had assumed RGB would be enough, but you can see the curves were less than ideal:
RGB curves will generally cover a larger gamut than CMY, and when including CMY and RGB, it smooths out the curves.
I will argue that white is not needed. The values used in Spectral.js are attenuated in the KM transformation and do not yield appropriate values. Reference issue where I originally brought a similar question: https://github.com/rvanwijnen/spectral.js/issues/17. I go into more specific details here. With all of that said, I don't think the inclusion of white impacts Spectral.js that much, as it clamps values to 8-bit sRGB.
The values for white are generated for GLSL, where they stay in linear space and there is no clamping to integer values. Without these specific white values, the mix will turn green over time (see this issue: https://github.com/patriciogonzalezvivo/lygia/issues/242).
I agree that this is not needed for the JavaScript library, or any other form that converts back to integer values, but I wanted the arrays to be the same for GLSL and JavaScript. There is more to it regarding the decisions I made, but, just like including CMY SPDs, it was necessary to make this work correctly.
As I said, spectral.js is not meant to be scientifically accurate, and I’d rather have slightly less correct white values than end up with green tinting.
I will state that the white values are not round tripping through the KM function. As a matter of fact, any time the sum is greater than 1, which happens as colors get closer to white, the values are become less accurate near white. Mathematically, any value over 1 is technically giving inaccurate results, and the white value indeed exceed 1.
(1 - R) ** 2 / (2 * R)
The clamping to 8-bit RGB minimizes these inaccuracies. Though I think that is where residuals can help, but with Spectral.js being 8-bit RGB, this becomes less important.
I also admit that I have not as of yet explored this repeated mixing scenario using residuals, so I can only say I think it intuitively would help, but I cannot say that with tests backing it up currently. I also admit Spectral.js may not need such a correction as it is confined to 8-bit RGB.
@rvanwijnen I took some time to dig into the green bias problem you linked, and I can confirm that including or excluding the white SPD has nothing to do with it (well, not exactly true, but it is not the critical piece to prevent such biases).
I took the Spectral.js Python version from v2.0.2 and updated it to the high-precision matrices and values used in v3. I also removed the expectation of RGB being scaled to 255 so I could use values in the range of [0, 1] directly. Everything else is identical. So, in this case, we are using high precision values, but using the old assumption of 1 for white. This is to reproduce the original issue.
I created a simple test to reproduce the green bias problem by following the link and using a color picker to select the lighter color: #A7C5BB.
>>> c1 = Color('#A7C5BB')
>>> colors = [c1]
>>> for _ in range(1000):
... c1 = Color('srgb', spectral_mix(c1.coords(), c1.coords(), 0.05))
... colors.append(c1)
...
>>> Steps(colors)
Using this, I was able to reproduce the green bias.
The reason this breaks is indeed because the assumption of white being 1 is wrong. The red, green, and blue curves do not sum to 1, so this introduces errors. So, using your corrected white values does help, but they are not needed as I will illustrate. What I mean by this is that white can be left out altogether, as the sum of the other channels is sufficient for white. The one caveat is that the algorithm that is currently used for concentration needs to be fixed.
In order to demonstrate this, I added two options: use_white and apply_fix. use_white will, of course, control whether the white data is used in the calculation (currently using old assumption of 1) and apply_fix will apply the concentration calculation fix.
Disabling white showed very wrong results, this is what tipped off that there was an issue with concentrations. These results indicate a fundamental problem with the concentration calculation.
When I ported spectral.js to my own library, I did not copy the concentration algorithm, but used a different approach that kept the spirit of what was being done. At the time, I did not realize that I was fixing an issue that existed in spectral.js and had assumed there would be no difference. It turned out that I had fixed an issue in Spectral.js without realizing it. Here is modified test code:
def spectral_upsampling(lrgb, use_white, apply_fix):
w = min(min(lrgb[0], lrgb[1]), lrgb[2]) if use_white else 0.0
lrgb = [lrgb[0] - w, lrgb[1] - w, lrgb[2] - w]
if apply_fix:
r, g, b = lrgb
c = max(min(g, b), 0.0)
m = max(min(r, b - c), 0.0)
y = max(min(r - m, g - c), 0.0)
r -= max(m + y, 0.0)
g -= max(c + y, 0.0)
b -= max(c + m, 0.0)
else:
c = min(lrgb[1], lrgb[2])
m = min(lrgb[0], lrgb[2])
y = min(lrgb[0], lrgb[1])
r = max(0, min(lrgb[0] - lrgb[2], lrgb[0] - lrgb[1]))
g = max(0, min(lrgb[1] - lrgb[2], lrgb[1] - lrgb[0]))
b = max(0, min(lrgb[2] - lrgb[1], lrgb[2] - lrgb[0]))
return [w, c, m, y, r, g, b]
When excluding white and applying the concentration fix, the green bias problem went away:
Now, does this break normal mixing? No:
Full example with test code available here: https://facelessuser.github.io/coloraide/playground/?source=https%3A%2F%2Fgist.githubusercontent.com%2Ffacelessuser%2F9a60f165a02d51539dbfd659ee073e60%2Fraw%2F8f7f2a5631ae262676d9c1989c8d84404465d320%2Fspectraljs.py
In short, including the better white data fixes the issue in older Spectral implmentations, but was masking a concentration calculation bug. Once the concentration bug is fixed, there is no need to include white data at all.
Hopefully this is found helpful.
This illustrates why Spectral.js's decomposition of Linear sRGB colors is not quite right. This is just showing how how Spectral.js is currently double counting channels when decomposing a color into concentrations for the primary wavelengths.
>>> color = Color('#A7C5BB').convert('srgb-linear')
>>> lrgb = color.coords()
>>> c = min(lrgb[1], lrgb[2])
>>> m = min(lrgb[0], lrgb[2])
>>> y = min(lrgb[0], lrgb[1])
>>> r = max(0, min(lrgb[0] - lrgb[2], lrgb[0] - lrgb[1]))
>>> g = max(0, min(lrgb[1] - lrgb[2], lrgb[1] - lrgb[0]))
>>> b = max(0, min(lrgb[2] - lrgb[1], lrgb[2] - lrgb[0]))
>>> print(
... '=== Spectral.js Approach ===\n'
... f'original lrgb: {lrgb}\n'
... f'cmyrgb: {[c, m, y, r, g, b]}\n'
... f'new lrgb: {[m + y + r, y + c + g, c + m + b]}'
... )
=== Spectral.js Approach ===
original lrgb: [0.386429433787049, 0.5583403896342679, 0.4969329950608704]
cmyrgb: [0.4969329950608704, 0.386429433787049, 0.386429433787049, 0, 0.0614073945733975, 0]
new lrgb: [0.772858867574098, 0.9447698234213169, 0.8833624288479194]
>>> r, g, b = lrgb
>>> c = max(min(g, b), 0.0)
>>> m = max(min(r, b - c), 0.0)
>>> y = max(min(r - m, g - c), 0.0)
>>> r -= max(m + y, 0.0)
>>> g -= max(c + y, 0.0)
>>> b -= max(c + m, 0.0)
>>> print(
... '=== New Approach ===\n'
... f'original lrgb: {lrgb}\n'
... f'cmyrgb: {[c, m, y, r, g, b]}\n'
... f'new lrgb: {[m + y + r, y + c + g, c + m + b]}'
... )
=== New Approach ===
original lrgb: [0.386429433787049, 0.5583403896342679, 0.4969329950608704]
cmyrgb: [0.4969329950608704, 0.0, 0.0614073945733975, 0.3250220392136515, 0.0, 0.0]
new lrgb: [0.386429433787049, 0.5583403896342679, 0.4969329950608704]
Live example here.
I don't know what you are trying to prove Isaac? In your example you didn't account for white, although you don't have to use an SPD for white you do need to include its weight.
>>> color = Color('#A7C5BB').convert('srgb-linear')
>>> o_lrgb = color.coords()
>>> lrgb = o_lrgb
>>> w = min(lrgb)
>>> lrgb = [lrgb[0] - w, lrgb[1] - w, lrgb[2] - w]
>>> c = min(lrgb[1], lrgb[2])
>>> m = min(lrgb[0], lrgb[2])
>>> y = min(lrgb[0], lrgb[1])
>>> r = max(0, min(lrgb[0] - lrgb[2], lrgb[0] - lrgb[1]))
>>> g = max(0, min(lrgb[1] - lrgb[2], lrgb[1] - lrgb[0]))
>>> b = max(0, min(lrgb[2] - lrgb[1], lrgb[2] - lrgb[0]))
>>> print(
... '=== Spectral.js Approach ===\n'
... f'original lrgb: {o_lrgb}\n'
... f'wcmyrgb: {[w, c, m, y, r, g, b]}\n'
... f'new lrgb: {[w + m + y + r, w + y + c + g, w + c + m + b]}'
... )
=== Spectral.js Approach ===
original lrgb: [0.386429433787049, 0.5583403896342679, 0.4969329950608704]
wcmyrgb: [0.386429433787049, 0.11050356127382138, 0.0, 0.0, 0, 0.0614073945733975, 0]
new lrgb: [0.386429433787049, 0.5583403896342679, 0.4969329950608704]
>>> r, g, b = o_lrgb
>>> c = max(min(g, b), 0.0)
>>> m = max(min(r, b - c), 0.0)
>>> y = max(min(r - m, g - c), 0.0)
>>> r -= max(m + y, 0.0)
>>> g -= max(c + y, 0.0)
>>> b -= max(c + m, 0.0)
>>> print(
... '=== New Approach ===\n'
... f'original lrgb: {o_lrgb}\n'
... f'cmyrgb: {[c, m, y, r, g, b]}\n'
... f'new lrgb: {[m + y + r, y + c + g, c + m + b]}'
... )
=== New Approach ===
original lrgb: [0.386429433787049, 0.5583403896342679, 0.4969329950608704]
cmyrgb: [0.4969329950608704, 0.0, 0.0614073945733975, 0.3250220392136515, 0.0, 0.0]
new lrgb: [0.386429433787049, 0.5583403896342679, 0.4969329950608704]
See here
White is just a flat line on the spectrum with the min value from lRGB, than subtract white for the rest and add on top (CMYRGB).
My original statement was white data is not needed. This isn't to say it cannot be used, but it turns out to be redundant.
I am simply clarifying my statement of why carrying around the white SPD values are not needed. You inherently have white already included when you sum RGB channels.
So, a fair point was raised about my example not subtracting white. So I will correct my statement. It would seem the current decomposition of linear sRGB expects white data to be subtracted, zeroing out at least one channel, and I will assume that if white is always considered, the results will always be correct.
In the context of excluding white data, a tweak is required to correctly decompose linear sRGB, and if that change is made, white data can be completely omitted from consideration, eliminating the need to carry around white SPD data.
Including white or not including white data (assuming the data is correct) is not what caused the green bias; using the previous assumption of 1 for the reflectance of white was. The approximation of RGB reflectance never summed to 1, and that assumption is what actually caused the original green bias in the linked issue.
Again, if you are happy including the white SPD in all calculations, that is fine; if you want to cut out some multiplications, you can drop the inclusion of white SPD data and tweak your concentration calculations.
The problem is here:
A flat SPD (all ones) does not map exactly back to the reference XYZ values when using the D65-weighted CIE 1931 2° observer. To address this, I computed a custom SPD for white that does reproduce the correct XYZ values. This corrected SPD is what’s included in spectral.js.
A flat SPD (all ones) does not map exactly back to the reference XYZ values when using the D65-weighted CIE 1931 2° observer. To address this, I computed a custom SPD for white that does reproduce the correct XYZ values. This corrected SPD is what’s included in spectral.js.
Right, and that's why you previously had the green bias, using flat 1. That's why Spectral.js v3 doesn't have issues, because it's using values that roughly equate to the sum of R, G, and B reflectance.
It seems I am not articulating myself well. I am not suggesting to use flat 1 SPD, I am suggesting to to use nothing for white and omit it completely. No white concentrations, no white SPD, no considerations of white at all. I've demonstrated in my early example that you get expected results without the green bias when omitting white completely. It's fine if you are against doing this, as using white can be done, as long as the values are accurate enough, I'm just stating you don't need white at all, and it would save calculations.
Mixbox included white because white was not a representation of other included pigments. It was a very particular shade of white with inherent reflective and scattering properties. But when you are considering light wavelengths, as is being done in Spectral.js, and when you consider the specific wavelengths that have been characterized, white is already represented as a combination of the existing wavelengths, and inclusion is redundant and unnecessary.
This is another example comparing corrected white SPD to no white at all. No green bias in either, color mixing looks the same, whites look the same, etc.
The green bias is only an issue in GLSL, it doesn’t occur in sRGB space.
I will test occluding white and also test the GLSL version with white occluded.
The green bias is only an issue in GLSL, it doesn’t occur in sRGB space.
It can with excessive, repeated mixing.
I will test occluding white and also test the GLSL version with white occluded.
I would just remember to adjust the concentration calculation so that it no longer expects white to be removed. Worst case, you find something I overlooked. Best case, you can remove a number of unnecessary multiplications.
Hi Isaac, I’ve tested this, and although omitting white is indeed possible, it introduces some caveats I’d prefer to avoid.
What you’re essentially doing with your method is adding cyan and red for the white channel. This creates a strange spike in the 500–600 nm range. It also sums to values above 1, just like my generated white SPD.
The difference is that my white SPD is smooth, I prefer that over the spiky one, even if it uses fewer calculations.
I have to admit that the errors are on such a minor scale that in practice it would probably not be noticeable.
I have created a Google Spreadsheet where you can check the calculations here
As I mentioned, either approach yields values above 1 and those break round trips through the KM function. But I agree that your white is more accurate in context to the other curves. I also agree that the difference is minimal enough that it probably isn't noticeable, but I can also understand the desire to lean into what is more "accurate".
The reality is that you will not get good roundtripping around white with either approach, nor will you get accurate values with some combinations of curves not specifically white when reflectance is greater than 1. The KM function just breaks down with those values.
The approximate calculations of the curves just don't work out such that the channels sum to 1 properly. This is likely due to the approach of the curve estimation evaluating each channel without the greater context. Red is not calculated with the context that whatever blue and green are, they must sum to 1 for white, etc. It's good enough though for what we want, but they are only an approximation.
To mitigate this error, I utilize the residual, but if Spectral.js dropped white to save some calculations, I realize now that you'd lose whatever you gained by calculating the residual. Since you are clamping to hex RGB anyway, the errors near white are likely mitigated.
Anyway, in short, I do understand where you are coming from 🙂.
I agree it's not ideal, but it’s necessary to prevent the green tinting when you stay in the lRGB space. If I were always clamping to 255 RGB values, I could just add the white value, save 38 multiplications, and stay within the 0–1 range.
I do think it’s best to treat white separately (instead of combining other channels), as it gives the smoothest curves.
I haven’t found an optimal solution for this.
I really enjoy our in-depth discussions, there aren’t many people we can talk to who have knowledge about this.
In the end, I think you won me over on the smoother curve argument. I think I will implement it with white to be faithful to this concept and to the Spectral.js implementation.
Correction to my earlier statement:
- I made it sound as if you dropped white, residuals would have to be used. Residuals are not specifically needed when dropping white. The KM function will not round-trip values greater than one, and inaccuracies near white and other places near the edge of the sRGB gamut will give less accurate results regardless of whether white is included or excluded. If you are confined to just the sRGB gamut, as Spectral.js is, these inaccuracies will not be visually noticeable. Residuals are a device to compensate for color quantities that cannot be accurately represented with the primary reflectance curves due to the reflectance exceeding one.
So in conclusion:
-
Including white gives better curves; differences in approach are likely not to be visually perceivable within the sRGB gamut, but including white offers a more sane approximation. And I agree gives a better approximation.
-
You can't get away with just CMY or just RGB. The approximations are not perfect, and including all (including white) helps to provide more information and smoother curves.
-
Including white or excluding white is not what caused the original green shift issue; it was specifically using a bad white that assumed a reflectance of 1 at every wavelength.
-
Regardless of the approach, the KM function cannot round-trip reflectance greater than one, but if colors are kept within sRGB, the results are close enough in most cases.
-
If you need better round-tripping of values where the gamut coverage due to reflection is not perfect (near white, for instance), or you were trying to handle colors beyond the primary reflectance curve's ability (wider color gamuts), residuals could be used to compensate for color concentrations that cannot be represented due to reflectance exceeding one, but this comes at a computational cost.
As a side note, for anyone wanting to calculate these original curves by hand:
-
White must be calculated using Scott Burns "LLSS" (Least Log Slope Squared) approach. Results from this match Spectral.js.
-
Other R, G, B, C, M, Y can use Scott Burns "LHTSS" (Least Hyperbolic Tangent Slope Squared) which will generate the curves keeping reflectance under one.