Add target HDR headroom to `CanvasRenderingContext2D `

[ccameron-chromium]

What is the issue with the HTML Standard?

There is no way to indicate what tone mapping, if any, is done when calling functions such as drawImage. Adding a targetHdrHeadroom attribute to one of the mixins of CanvasRenderingContext2D would address this problem.

See explainer at: https://github.com/ccameron-chromium/ColorWeb-CG/blob/master/canvas2d_hdr_headroom.md

[css-meeting-bot]

The CSS Working Group just discussed Add target HDR headroom to `CanvasRenderingContext2D ` .

The full IRC log of that discussion

<annevk> [Not minuting as people can just read the proposal.]
<annevk> weinig: So this controls the HDR headroom for input into the canvas?
<annevk> ccameron: Yup.
<astearns> Sam’s proposal for definitions to spec against: https://github.com/w3c/csswg-drafts/issues/11307#issuecomment-2718858571
<annevk> What's color-hdr?
<astearns> (our draft shortname, but annevk dropped) https://drafts.csswg.org/css-color-hdr-1/
<dholbert> hmm, https://drafts.csswg.org/ is loading slowly today; getting hammered by crawlers or something?

[annevk]

Where is color-hdr defined?

[annevk]

cc @whatwg/canvas

[ccameron-chromium]

Where is color-hdr defined?

That's a proposal from https://github.com/w3c/csswg-drafts/issues/11616. It's in the spec at https://drafts.csswg.org/css-color-hdr/#hdr-color-function.

[annevk]

I think you can consider WebKit interested in this. Some thoughts and questions:

1. How does this relate to the CSS property for HDR content?
2. How do 0 and 1 differ?
3. How will NaN and negative numbers be handled?
4. Per https://w3ctag.github.io/design-principles/#casing-rules it has to be targetHDRHeadroom.

[ccameron-chromium]

1. How does this relate to the CSS property for HDR content?

This is related to how content gets draw into the canvas' backing bitmap. The CSS dynamic-range-property is related to how that bitmap gets rendered to the screen.

So they are in theory completely separate. In practice, people will likely use them together.

2. How do 0 and 1 differ?

It looks like we're going to use log2-scale for HDR headroom across HTML and CSS (which is totally fine). That means that "0" is SDR, and "1" is "you can get 2x brighter in linear space".

3. How will NaN and negative numbers be handled?

Throwing an exception seems appropriate to me.

4. Per https://w3ctag.github.io/design-principles/#casing-rules it has to be targetHDRHeadroom.

SGTM.

One thing to think about is that, despite the fact that we've both landed on "target" in our implementations, do you think drawingHDRHeadroom would be good? It might help head-off confusion about the "drawing to the bitmap" versus "displaying the bitmap to the screen". It also aligns with the all the "drawing" text in the spec, and some of the "drawing" functions (like drawImage, but it also applies to fillRect and strokeRect, etc).

[annevk]

Agreed that we don't have to match those APIs naming-wise. Maybe even just hdrHeadroom? It being on the context should provide enough clarity on its own.

So if people wanted their HDR image to get rendered identically through <img> and drawImage(), they can always accomplish that with Infinity? Or are they going to run into a math problem? (For images with gain maps we'd need new functionality.)

Presumably the setter should also throw when the colorType is not "float16"?

[Kaiido]

Just to participate in the bike-shedding, what about globalHDRHeadroom? This would match globalCompositeOperation & globalAlpha.

[ccameron-chromium]

So if people wanted their HDR image to get rendered identically through <img> and drawImage(), they can always accomplish that with Infinity? Or are they going to run into a math problem?

Unfortunately it's impossible for <img> to match drawImage() for HDR content in the general case.

The only case where it would be possible would be to draw with Infinity, and then attach to the canvas, for rendering, the same HDR metadata that the original image had.

Presumably this property should also throw when the colorType is not "float16"?

For the srgb and display-p3 color spaces, that could be an option, but for things like rec2100-hlg, 8 bits can be desirable. So we'd need to have a list of which (color space, color type) pairs are reasonable, which might get involved.

[ccameron-chromium]

Just to participate in the bike-shedding, what about globalHDRHeadroom? This would match globalCompositeOperation & globalAlpha.

It's true that this is similar in behavior to globalAlpha. Is there any documentation about what the "global" thing is. I vaguely prefer globalTargetHDRHeadroom, but maybe it's just because I just haven't become okay with globalHDRHeadroom.

[Kaiido]

I couldn't find an actual "documentation" about that wording, but it seems it was in the first version ever of Web Apps that introduced the canvas 2D API (I feel like I'm covered in dust digging this up). At that time there was a sentence stating:

All drawing operations are affected by the global compositing attributes, globalAlpha and globalCompositeOperation.

So I guess the idea was that since these do affect all drawing ops, they're "global".
This mailing thread also had the same argument for imageSmoothingEnabled at the time it was thought it could apply to all drawings, though it ended up affecting only drawImage and CanvasPattern.

[greggman]

I'm sorry I'm not familiar with all of the issues here but ... what do I do with this number? How do I know what to set it to?

Let's say I want to make a photo editing app. This is probably hopelessly naive but

• I want to make a lossless HDR photo editing app
• I want to use Canvas/WebGL/WebGPU
• I want to import images from any source

I would think the data in my "document" (canvas?) would be in some "ideal" floating point format that represents physics or something. I import all images with a conversion to this "ideal" format. If possible, I put this data in a floating point canvas and I tell the browser how to display this ideal format (css?). Or, I have to do some conversion from ideal format to a floating point canvas. I save to this ideal format (think .PSD as in a file only useful my app). I export to other formats (.PNG, .JPG, .JPG with gain map, .HEIF, ...)

Where does hdrHeadroom fit in this?

[mwyrzykowski]

Where does hdrHeadroom fit in this?

My understanding, please correct me if this is not the case, there are use cases where the author wants their canvas content to display less bright than the screen supports. E.g., a thumbnail photo gallery.

So I set an hdrHeadroom of 1.0 on my thumbnails so my HDR thumbnails are no more than 2x brighter than SDR. Assuming the display supports 16x brighter than SDR, this avoids the thumbnails appearing incredibly bright while still showing a range of values greater than the SDR range.

[Kaiido]

there are use cases where the author wants their canvas content to display less bright than the screen supports.

This would probably be better handled by CSS, https://drafts.csswg.org/css-color-hdr/#controlling-dynamic-range

I want to make a lossless HDR photo editing app

For this case I suppose you should use Infinity, as in, "keep whatever value the input had".

In practice, I suppose 0 (SDR) and Infinity (source-HDR) will be the most used values in the wild. But as was explained to me, there are indeed cases where a boolean wouldn't suffice.
Having a variable value in the canvas context is required when you mix multiple sources with different headrooms onto the same canvas. The most extreme case being SDR and HDR, but even multiple HDR will have different headrooms. This somehow involves you know the innate headroom of each sources though.

[reitowo]

https://issues.chromium.org/u/1/issues/435875063

I also went into this issue when I'm trying to draw <img> element on a HDR canvas, the image's HDR metadata that embeds into the file works well with <img>, but when call drawImage these thing are discarded and the outcome image is wrong. I think a hdrHeadroom or this issue may not solve this problem, right? 1) Nowhere to get this from <img>. 2) hdrHeadroom is not keeping everything of hdr metadata.

Is it possible to let drawImage use <img> 's internal hdr metadata to automatically do some mapping? This actually seems like a bug to me. After mapping/rectifying <img> to context, maybe the data we get from getImageData will have correct output?

Repro: https://rsqjmy-5173.csb.app/

[smfr]

Just to participate in the bike-shedding, what about globalHDRHeadroom? This would match globalCompositeOperation & globalAlpha.

It's true that this is similar in behavior to globalAlpha. Is there any documentation about what the "global" thing is. I vaguely prefer globalTargetHDRHeadroom, but maybe it's just because I just haven't become okay with globalHDRHeadroom.

Given that mixin CanvasState is currently just functions, I feel like this is more appropriate for mixin CanvasCompositing.

[ccameron-chromium]

Given that mixin CanvasState is currently just functions, I feel like this is more appropriate for mixin CanvasCompositing.

Agree. Updated the explainer to add globalHDRHeadroom to CanvasCompositing.

[ccameron-chromium]

I've moved the "linear versus log2" discussion to https://github.com/w3c/ColorWeb-CG/issues/129, which needs to decide it for CSS/2D/WebGL/WebGPU. The direction appears to be "linear is preferred", but I want to make sure all APIs cover this. (And I'm copy-pasting this message into a handful of different bugs on this issue, sorry for the spam if you see it twice).