`colorMetadata` should not be limited to rendering to a display

[palemieux]

See https://github.com/whatwg/html/issues/9461#issuecomment-1611015012

[Kaiido]

colorMetadata is intended to be used whenever the image is rendered, i.e. the pixels of the image transformed, and is not limited to rendering to a physical display. I will file an issue to clarify this.

Originally posted by @palemieux in https://github.com/whatwg/html/issues/9461#issuecomment-1611705004

So if I understand correctly, this would enter the unclear and problematic "painting time", right?. For instance a series of drawing operations may not trigger that "rendering" up until the canvas is drawn on the screen, or on another canvas, or until it's read through getImageData(), or createImageBitmap(), etc.
Once again, web authors can't really know when this "rendering" will happen exactly, so giving them the ability to change these settings "on the fly" seems problematic, as they can't really know when to unset it.

[palemieux]

I do not understand what problem(s) this causes. Do you have a concrete example or a similar situation in a different context?

An author can change properties of the DOM dynamically, e.g. background color. Isn't that similar?

[Kaiido]

Maybe I'm missing a big point here, and if it's the case I'm sorry. This option will affect the resulting pixels values on the canvas buffer, right? I.e. the ones that can be read through getImageData()? It's not supposed to be used only by the web-renderer or embedded as a metadata in exported files?

If my understanding is correct, then my concerns are on the "when" this attribute would apply.
From your explanation it would seem that it only applies when the canvas is being "rendered", which is actually relatively undefined as of today.

ctx.colorMetadata = metadata1;
ctx.drawImage(imgA, x, y);
// some random time later
ctx.colorMetadata = metadata2;
ctx.drawImage(imgB, x, y);

In above example, depending on what happens during // some random time later the canvas could have been rendered using metadata1, or not, based on currently undefined or suprising conditions. For instance if the browser did paint the canvas on the screen during that time, imgA will be rendered using metadata1, but if it didn't, it will use metadata2 instead. But even things like anotherCtx.drawImage(ctx.canvas, x, y) would also force this rendering, synchronously, which is surprising.

I guess my point is that the Canvas2D API currently doesn't rely on this "rendering time" at all, and all its attributes are applied at the time of "drawing" (i.e. when the drawing commands are to be executed, as opposed as to when the bitmap is "rendered"). If you look for instance at how the compositing rules are defined currently, each new drawing will use the compositing rule that was set at the time it was called. Having an attribute that has its effects at some random time, sounds a bit concerning.

One possible way around that would be to limit this setting to inside layers, the same way it's being proposed for filters. This way the author would have control over what content is affected by this attribute. But this would probably delay the landing of the feature since the layers proposal seems a bit stalled currently.

[palemieux]

Wouldn't it be "surprising" only if the author changes the value of colorMetadata after setting it initially.

The primary use case for colorMetadata is to prevent fluctuations in the tone-mapping algorithm across multiple images, so I would expect it to remain unchanged.

Makes sense?

[ed.: thanks for walking me through this.]