[css-color-4] Disagreements over gamut mapping

[sesse]

There's been some discussions around gamut mapping: https://csswg.sesse.net/css-color-4/#css-gamut-mapping

The questions in point are:

• Is §13.2 normative? There have been discussions about the use of the phrasing “This CSS gamut mapping algorithm” versus “The CSS gamut mapping algorithm”, and the non-normative marker on §13.1 (does it extend to all of §13?). Must out-of-gamut colors be mapped, using that specific algorithm, on both conversion-to-HSL and for display, to be compliant with CSS Color Level 4?
• What, if anything, should be done with out-of-gamut colors in images and videos? Do we risk that using the same color in e.g. CSS and on an image mapped to different colors on the users' screens, and is that okay?
• What about HDR images and video; are those subject to the same rules, different rules, or undefined?

[svgeesus]

Is §13.2 normative? There have been discussions about the use of the phrasing “This CSS gamut mapping algorithm” versus “The CSS gamut mapping algorithm”, and the non-normative marker on §13.1 (does it extend to all of §13?).

13.1 is, as stated, a non-normative introduction to the topic. 13.2 is normative.

This CSS gamut mapping algorithm

Thanks for pointing out the possible ambiguity, I agree that The would be better.

Must out-of-gamut colors be mapped, using that specific algorithm, on both conversion-to-HSL and for display, to be compliant with CSS Color Level 4?

Yes. HSL (and HWB) are unable to represent out-or-sRGB-gamut colors.

What, if anything, should be done with out-of-gamut colors in images and videos? Do we risk that using the same color in e.g. CSS and on an image mapped to different colors on the users' screens, and is that okay?

"If anything"? Colors which are out of the display device gamut cannot be displayed (by definition).

Gamut mapping for images (and video) has different constraints to solid colors, such as preservation of image detail. For photographic images, a perceptual rendering intent should preserve most of the overall image appearance but may change in-gamut colors. For non-photographic images such as charts and diagrams, a relative colorimetric intent will give more similar results to the CSS GMA (but not identical, because the ICC code path will be using CIE Lab as the gamut mapping space). Yes, that means that out of screen gamut colors may map differently in images and in CSS, particularly wildly OOG colors such as those outside the spectral locus. These are much less likely in photographic images but can occur in synthetic images. It isn't desirable but there isn't a better option because relative colorimetric handling of photographic imagery gives terrible results.

What about HDR images and video; are those subject to the same rules, different rules, or undefined?

The same "this is a different code path" rules i.e. not the same as CSS. In addition to the GMA there will also be tone mapping, unless the display has the same peak white as the mastering display. All of that is generally a black box, especially for HDR video.

[sesse]

Yes. HSL (and HWB) are unable to represent out-or-sRGB-gamut colors.

Sure, but for display, does the spec require this specific algorithm and not e.g. just clipping each color component to [0,1]?

[ccameron-chromium]

Suppose a page specifies the color "color(display-p3 1 0 0)", and then has an image element with an image in Display P3 color space, with pixel value (1, 0, 0). Should those two colors be identical when displayed on-screen?

[ccameron-chromium]

Similarly to the above, what if both CSS and a 2D canvas use the color "color(display-p3 1 0 0)"? What about a WebGL canvas that writes a pixel value (1, 0, 0) and has specified its color space to be "display-p3".

I believe that it is a goal to allow color matching between CSS, images, and canvases (ideally video, too, but there is longstanding historical divergence in interpretations of rec601 and rec709 -- something for another discussion -- I'd love to resolve that, maybe WebCodecs will let us).

In order to guarantee color matching between CSS and images, it is necessary that the same processing be applied to pixels coming from CSS and pixels coming from images. This precludes a scheme of "apply gamut mapping to just CSS colors and not (necessarily) images".

This can be fixed by gamut mapping images and canvases. That, however, comes at a significant performance cost. For CSS colors, there are lots of places in the pipeline to allow injection of various transforms at very little (or no) extra cost. For canvas elements, they are often handed directly to the display controller, which has only fixed-function hardware for color management, and is incapable of performing gamut mapping. The hardware is evolving, and there is a big push to support per-plane 3D LUTs (which would allow hardware tone mapping and hardware gamut mapping), but I'm not sure they exist in shipping devices (let alone are universal).

If we were to require gamut mapping for canvases, it would mean that no canvas that can express any color value outside of the gamut of the display can be represented as a hardware overlay. That would be catastrophic for battery life, especially on mobile devices. (There's almost no limit to the work that we will do to keep a buffer in a hardware overlay plane).

This line of reasoning leads me to the position that gamut mapping should be put as the responsibility of the underlying operating system, display controller, and even display device(!). For the next few years (until display controllers all have 3D LUTs, completing yet another turn of the wheel of reincarnation), this will mean that out-of-gamut colors may not look as good as they could on some devices. Content authors for whom this is a matter of grave concern may use the media queries (or the exact query of primaries, which is part of the HDR canvas proposal), to ensure they do not serve content outside of the gamut of the target device.

[sesse]

Cc-ing @weinig, since he seems to have done the work on gamut mapping in WebKit.

FWIW, I don't think “someone makes a DCI-P3 canvas or WebGL context on an sRGB device” is a use case that should be driving for our general color decisions. It seems very niche compared to the amount of pages that simply use CSS colors and regular images (or sRGB canvases). And AIUI, “colors won't look good on some devices” essentially means “on every non-Apple device in existence” for at least the next couple of years…

[foolip]

@weinig might you be able to summarize the "architecture" of color handling in Safari?

From testing in Safari Technology Preview 151 it's clear that something has changed from the previous behavior of clamping each component separately. One of the test cases that @sesse used was color(srgb 2 0 0) which in Safari stable clamps to color(srgb 1 0 0) in computed style, but in STP remains as color(srgb 2 0 0) and renders as something close to color(display-p3 0.9992 0.4399 0.3215) according to Web Inspector's color picker.

It would be great to understand a bit about where in the pipeline such mapping happens, and whether it only applies to out-of-gamut cases like color(srgb 2 0 0), or if something similar would also happen when color(display-p3 1 0 0) is to be displayed on an RGB monitor?

[ccameron-chromium]

Bear in mind that the color (srgb 2 0 0) is not just out-of-gamut, but also has a luminance that is out of the SDR luminance range. That's likely the reason why it is being bent towards white.

[foolip]

@ccameron-chromium good point, it's also worth testing somewhere where luminance is in SDR range. I think color(srgb 1.05 0 0) is such an example, which STP renders as color(display-p3 0.9639 0.2133 0.1489). I'm not sure what to conclude from this, except that it's not merely clamping to RGB red, as Safari stable did.

[facelessuser]

good point, it's also worth testing somewhere where luminance is in SDR range. I think color(srgb 1.05 0 0) is such an example, which STP renders as color(display-p3 0.9639 0.2133 0.1489). I'm not sure what to conclude from this, except that it's not merely clamping to RGB red, as Safari stable did.

It's not clamping to sRGB Red, it is clamping in the Display P3 color space. It isn't doing any fancy gamut mapping, just simple clipping in Display P3, as most new Macs have Display P3 monitors. Safari, due to its "color management" then works in Display P3. I guess this is new behavior between stable and STP. That's all that happening.

>>> Color('color(srgb 1.05 0 0)').convert('display-p3').clip()
color(display-p3 0.96358 0.21239 0.14773 / 1)

As far as I can tell, Safari doesn't do anything but simple clipping currently. On Display P3 systems, it will clip to the Display P3 color space, and on sRGB systems, it will clip to sRGB. We can see this by using color(srgb 2 0 0).

>>> Color('color(srgb 2 0 0)').convert('display-p3').clip()
color(display-p3 1 0.44226 0.32203 / 1)

If it was doing gamut mapping as described in the CSS spec, the results would probably be white as the color would be beyond the luminance of Display P3. The CSS spec currently recommends gamut mapping in Oklch, so if we look at the slice of Oklch in which gamut mapping would occur, limiting the gamut to Display P3, we can see why it would go to white.

>>> Color('color(srgb 2 0 0)').convert('display-p3').fit(method='oklch-chroma')
color(display-p3 1 1 1 / 1)

EDIT: I don't have anything to do with Safari, and this is all just based on simple observation. I don't know if they have more advanced gamut mapping planned or not.

[sesse]

When I tested something like color(srgb 10 0 0) in Safari TP, it would go towards a very pale yellow on the internal Display-P3 screen. This is incompatible with the notion of a simple clipping in DCI-P3 space.

[facelessuser]

That is not what I see at all in Safari TP, I get white on my Display P3 monitor. Unless there is some other special feature you have enabled that I don't.

This makes sense as color(srgb 10 0 0) is so far out in the land of imaginary colors (past the spectral locus) that all components exceed 1 and it just gets clipped to white. Even gamut mapping will leave you with just white. You can try it here.

[sesse]

But that is, indeed, incompatible with the theory that it's just clipping each component in display-p3.

Anyway, let's wait for the people who actually implemented this to chime in?

[svgeesus]

In order to guarantee color matching between CSS and images, it is necessary that the same processing be applied to pixels coming from CSS and pixels coming from images. This precludes a scheme of "apply gamut mapping to just CSS colors and not (necessarily) images".

Feel free to test that out with some photographic images (you will need to override the rendering intent in the ICC profile, which will likely be set to perceptual).

Wanting solid colors and images to gamut map the same is a great goal, and sounds reasonable until one looks into what is currently done for gamut mapping images.

For an in-depth overview, I recommend Color Gamut Mapping by Ján Morovič

[svgeesus]

“colors won't look good on some devices” essentially means “on every non-Apple device in existence” for at least the next couple of years…

Yes but to clarify: there are plenty of non-Apple devices (laptops, tablets, phones) with WCG (P3-ish or Adobe RGB-ish) screens. It is not the hardware that is lacking, but WebKit browsers being confined to Apple hardware, and non-WebKit browsers being confined to sRGB.

[facelessuser]

But that is, indeed, incompatible with the theory that it's just clipping each component in display-p3.

It's not really a theory, and I'm not following the above statement as getting white in TP in this scenario shows this exactly, along with the other conversions I was able to replicate with simple clipping. I've actually looked into this extensively in the past.

Anyway, let's wait for the people who actually implemented this to chime in?

That's fine :shrug:. Just thought I'd try and save you some time.

[facelessuser]

I should probably state that everything I've said only applies to colors, not images and such. I've done no comparisons as to what any browser does with images or videos.

[facelessuser]

One other clarification, my statements are also based on the idea that a Display P3 monitor is using a Display P3 color profile. If you were using a different color profile for your monitor, your color results will be different. You can actually try this out and see the differences.

[weinig]

@weinig might you be able to summarize the "architecture" of color handling in Safari?

Not sure I can comprehensively summarize the architecture of color handling in WebKit in a succinct manner, but I can explain what our current behavior and longer term intentions are with out-of-gamut colors.

Currently, WebKit keeps colors in their described form (so, for instance, color(srgb 2 0 0) keeps the 2) up until use time, so that means the computed values should show the same out of gamut values. Then at use time, we paint into a context and allow the platform to perform its default gamut mapping.

A change we plan to make is that instead of using the platform specific gamut mapping, we are going to use the CSS Gamut mapping algorithm (https://drafts.csswg.org/css-color-4/#css-gamut-mapping). We are still trying to determine whether it makes sense to gamut map to the exact color space of the underlying context (usually the color space of the display) or to instead pick a color space based on the gamut matched by the color-gamut media query. So for instance, for a color used on a display with a gamut at least as big as Display-P3, we would use the CSS Gamut mapping algorithm to map to Display-P3. I think the first approach (mapping using the CSS Gamut mapping algorithm to the display's color space) would be preferred, as that would more closely match images, but it has some drawbacks due to not being consistent across machines.

[weinig]

I should also add that we already apply the CSS Gamut mapping algorithm in WebKit in at least one script observable place (the ones in the previous comment are not), by using the color-mix function with the hwb or hsl color interpolation method.

For example, if you have:

color-mix(in hsl, color(srgb 2 0 0) 50%, color(srgb 0 0 2) 50%)

We first gamut map the two inputs into the sRBG gamut, and then mix them. This is specified here: https://drafts.csswg.org/css-color-5/#color-mix-result

[ccameron-chromium]

I want to get back to the color matching issue, and take the image part out of the equation.

Consider the following 3 pieces of web content:

• A div which uses color(display-p3 1 0 0) as the background
• A 2D canvas in display-p3 in which solid color(display-p3 1 0 0) is drawn
• A WebGL canvas in display-p3 which is cleared to the color (1, 0, 0, 1)

Should these appear the same when displayed on a device that has a gamut that is, say, halfway between sRGB and P3? If not, why not?

If a content author wants to display something that part-div, part-2D-canvas, and part-WebGL, should it be possible for the author to color-match between these elements, so that there are no seams in their content, or not?

[foolip]

Thanks you @weinig, that's very helpful! You mention that computed values are unchanged, and there's a bit of https://drafts.csswg.org/css-color-4/#color-function that I'd like your take on:

An out of gamut color has component values less than 0 or 0%, or greater than 1 or 100%. These are not invalid; instead, for display, they are css gamut mapped using a relative colorimetric intent which brings the values within the range 0/0% to 1/100% at computed-value time.

Following the spec on this point would make the result of gamut mapping visible via getComputedStyle(element). Do you think the spec should change on this point?

[foolip]

@ccameron-chromium my 2c is that making those the 3 cases display the same color is important, and hopefully there is a solution that preserves this in most or all situations. I think gamut mapping to match the actual display is the "risky" operation here, if that's done for color(display-p3 1 0 0) before painting the div that would lead to a mismatch between that div and the 2D canvas.

To me this suggests that gamut mapping to match the actual display should happen late in the pipeline, unobservable to web contents, including when reading back pixels from a canvas.

It's still necessary to do something with color(srgb 2 0 0) whether that ends up painted to an sRGB or P3 buffer, and the color matching goal doesn't tell us what that something is, I think?

[sesse]

We are still trying to determine whether it makes sense to gamut map to the exact color space of the underlying context (usually the color space of the display) or to instead pick a color space based on the gamut matched by the color-gamut media query. So for instance, for a color used on a display with a gamut at least as big as Display-P3, we would use the CSS Gamut mapping algorithm to map to Display-P3. I think the first approach (mapping using the CSS Gamut mapping algorithm to the display's color space) would be preferred, as that would more closely match images, but it has some drawbacks due to not being consistent across machines.

Would this be script-visible, or just for display?

If it's script-visible, it would sound better to map to the “ideal” space (sRGB or DCI-P3), so that you get consistent results from machine to machine (less confusing, easier to test in WPT, less fingerprint risk). If it's only about display, I may have (weak) opinions but I think it's out-of-scope for the spec, and probably subtle enough that either way is fine.

[LeaVerou]

In addition to @svgeesus' points, there are different performance allowances to gamut mapping the relatively few CSS colors defined in stylesheets (even with interpolation) compared to gamut mapping every pixel on an image or video. For CSS colors, we can afford to prioritize getting a better color even if that's not realistic for perf reasons for images or videos.

There's been some discussions around gamut mapping: csswg.sesse.net/css-color-4/#css-gamut-mapping

Is there a reason you are linking to your own version of the spec? At first I thought it was done to include commentary as annotations but I don't see any (unless I missed it).

[sesse]

Is there a reason you are linking to your own version of the spec? At first I thought it was done to include commentary as annotations but I don't see any (unless I missed it).

drafts.csswg.org was down. It's a pure mirror (updated every night from GitHub).

[weinig]

Thanks you @weinig, that's very helpful! You mention that computed values are unchanged, and there's a bit of https://drafts.csswg.org/css-color-4/#color-function that I'd like your take on:

An out of gamut color has component values less than 0 or 0%, or greater than 1 or 100%. These are not invalid; instead, for display, they are css gamut mapped using a relative colorimetric intent which brings the values within the range 0/0% to 1/100% at computed-value time.

Following the spec on this point would make the result of gamut mapping visible via getComputedStyle(element). Do you think the spec should change on this point?

Oh, very interesting. I don't believe that was in the spec when I was last working on this (I made an intentional change to match the spec and to not clamp around a year ago if I remember correctly), so thank you for bringing this to my attention.

@svgeesus @LeaVerou, what was the driving motivation behind this change? To me, it has some unfortunate downsides, and I am not clear what you gain from it over the (or perhaps just my) previous interpretation which was that out-of-gamut colors would be preserved all the way to use time. (the benefits being that values round trip cleanly and that you can use things like out of gamut color(srgb ...) to express things like Display-P3 colors, which is a common practice in Apple's graphics stack these days).

Ultimately, if the spec authors think this behavior is preferable, this is an easy thing for us to implement (we used to do a clamping gamut mapping here after all, I could just plug in the CSS Gamut Mapping algorithm instead) and I will be happy to.

One thing to consider, once the use cases are better understood, is whether it makes sense to have both variants, bounded and extended. In Apple's graphics stack, we have both for all RGB like color spaces, kCGColorSpaceSRGB which is bounded and kCGColorSpaceExtendedSRGB which is unbounded (actually, there are four, bounded and gamma encoded, bounded and linear, extended and gamma encoded, extended and linear, and I have expressed some interest in the past in considering how we might be able to provide ways to express all of these in CSS, for instance color(srgb-bounded ...), color(srgb-bounded-linear ...), color(srgb ...), color(srgb-linear ...)).

[weinig]

We are still trying to determine whether it makes sense to gamut map to the exact color space of the underlying context (usually the color space of the display) or to instead pick a color space based on the gamut matched by the color-gamut media query. So for instance, for a color used on a display with a gamut at least as big as Display-P3, we would use the CSS Gamut mapping algorithm to map to Display-P3. I think the first approach (mapping using the CSS Gamut mapping algorithm to the display's color space) would be preferred, as that would more closely match images, but it has some drawbacks due to not being consistent across machines.

Would this be script-visible, or just for display?

If it's script-visible, it would sound better to map to the “ideal” space (sRGB or DCI-P3), so that you get consistent results from machine to machine (less confusing, easier to test in WPT, less fingerprint risk). If it's only about display, I may have (weak) opinions but I think it's out-of-scope for the spec, and probably subtle enough that either way is fine.

The intention is for it to not be script visible, as this would, as you note, would allow using the display's color space. There are few places where getting the actual color space of the display is challenging from the engine (though not insurmountable), so I think having the guarantee be a little looser and match the color-gamut media query would be be preferable. Ultimately, I don't think authors relying on colors outside the gamut that the color-gamut media query resolves to is all that useful a thing for authors (e.g. how important that on a display that has a gamut between P3 and Rec2020, out of P3 gamut colors get fully realized?) to do, but I could be convinced otherwise by compelling use cases.

[svgeesus]

Following the spec on this point would make the result of gamut mapping visible via getComputedStyle(element). Do you think the spec should change on this point?

Oh, very interesting. I don't believe that was in the spec when I was last working on this (I made an intentional change to match the spec and to not clamp around a year ago if I remember correctly), so thank you for bringing this to my attention.

@svgeesus @LeaVerou, what was the driving motivation behind this change? To me, it has some unfortunate downsides, and I am not clear what you gain from it over the (or perhaps just my) previous interpretation which was that out-of-gamut colors would be preserved all the way to use time.

My recollection is that CSSWG went back and forth on this, between computed value time and used value time; and the main driver was handling system colors in forced color mode, and handling currentColor.

I think that mapping to the display gamut should be as late as possible, so that out of gamut values like rgb(100% 100% -34.627%) (which is color(display-p3 1 1 0)) round-trip cleanly, preserving author intent.

I would need to dig into the commit history to see when that was changed and in response to what issue, but from memory it was to do with handling system colors.

[svgeesus]

drafts.csswg.org was down. It's a pure mirror (updated every night from GitHub).

It is very useful to have a reliable mirror, thanks for that!

[svgeesus]

Ultimately, I don't think authors relying on colors outside the gamut that the color-gamut media query resolves to is all that useful a thing for authors (e.g. how important that on a display that has a gamut between P3 and Rec2020, out of P3 gamut colors get fully realized?)

We will see this increasingly, as video content creators extend beyond DCI-P3 as a mastering volume and consumer displays (mainly TVs, to start) push beyond P3. The actual content is delivered in a BT.2020 or BT.2100 container, but does not use the fulll 2020 gamut. And displays will not go all the way to full 2020 for some time, because of the issues of luminous efficiency, speckle, and strong observer metamerism for genuinely single-wavelength display primaries.