Take visual viewport (pinch-zoom) into account

[page200]

There's no pixel perfection when window.visualViewport.scale isn't 1, for example "when the user has pinched-zoomed" on a touchscreen or touchpad.

This requires two changes:

• https://github.com/greggman/pixel-perfect.js/blob/65b1608868bbc882e98b96784a721bbbb5b1ba55/pixel-perfect.js#L69 doesn't trigger when the visual viewport changes and we additionally need window.visualViewport.addEventListener('resize', ...)
• So far, devicePixelRatio was used both to achieve pixel perfection if window.visualViewport.scale==1 as well as choose the image size based on the user's zoom wishes expressed via Ctrl++. Replacing devicePixelRatio by devicePixelRatio * window.visualViewport.scale would achieve pixel perfection even if window.visualViewport.scale isn't 1 as well as choose the image size based on the user's zoom wishes expressed via Ctrl++ as well as via pinch gestures.

[greggman]

Thanks!

Unforutnately using viewViewport.scale does NOT achieve pixel perfection. Just multiplying by scale doesn't choose the correct size :(

[page200]

I guess you mean visualViewport.scale. What size does it choose if not the correct one? If you see a discrepancy between the actual scale and the scale reported by visualViewport.scale, logging how that discrepancy depends on the scale, device, and browser might help us (or answerers on Stack Overflow) understand the reasons.

Checking how others use visualViewport.width visualViewport.scale in practice (also in combination with devicePixelRatio) might also help. For example:

• https://github.com/BlobTheKat/blob-machine/blob/8630b5effe1fca0c6d64b57ef1eefcdc624ed605/index.html#L60-L61
• Post from October 16, 2022 at https://t.me/s/SmartHypercube_channel (I quote it below without indentation because indentation messes up the numbering).

浏览器里的 canvas 好难用。说两个听起来很简单但实际上很难实现对的需求：

1. 让 canvas 的像素和物理像素一一对应，来确保绘制出的图像是像素级完美的。很多人知道 devicePixelRatio 的概念，以及 1px 并不真的是一个物理像素，但不少网站错误地假设 devicePixelRatio 一定是整数。还有一个概念叫 visualViewport.scale。1px 其实对应 devicePixelRatio * visualViewport.scale 个物理像素。

2. 在 1 的基础上，确定鼠标所在的精确物理像素，以便在鼠标所在处绘制内容。鼠标相关事件中给出的几种坐标都是以 px 为单位的整数，由于 1px 不是一个物理像素，鼠标肉眼可见地移动了几个像素时，JS 获取到的坐标经常并没有变化，相关信息在取整后丢失了。screenX 和 screenY 是以物理像素为单位的，不会丢失信息，但并不容易把它换算到 canvas 坐标系，我还没发现任何可靠的方法获取 canvas 在 screen 上的物理像素坐标。

[greggman]

It's got nothing to do with how other people use it. It's got to do with testing it and seeing that it doesn't work. You do this by making an image in device pixels that will show a morie pattern if the calculation isn't perfect.

Here's an example: https://greggman.github.io/doodles/test/devicepixelcontentbox-with-visualviewport-scale/dpr-zoom-test-with-visualviewport.html

If the calculation is perfect then you should see solid stripes, orange and cyan. If the calculation is not perfect then you'll see a gradient of color (orange to dark orange), cyan to dark cyan). If you turn on image-rendering: pixelated then you'll see some line is doubled.

Here's a video: Unforunately, compression artifacts might not make it clear things are broken. But you can run the test yourself. Open the test above on a Mac. See, by default, the colors are solid. Pinch to zoom on the trackpad. See the colors stop being solid (because devicePixelContentBox * visualViewport.scale does not return the actual number of device pixels so the page can't correctly render a perfect match for the device)

https://github.com/user-attachments/assets/6ff5b48d-2842-47f4-a64f-f9f41a871a70

note that in the video, switching to ceil or round looks better but if you look closely you'll see the color on the right is not the same as the color on the left. This is more easy to see if you do it live.

[page200]

Thank you for the example. With it, I get pixel perfection with pixelated in Chrome and Firefox on Windows most of the time. Rarely, I get stripes like these with ceil or floor:

(You might want to view the image with a pixel-perfect viewer.)

The fact that some (red) stripes don't go all the way might be some kind of clue.

When there are these errors and I turn off pixelated, the bad stripes disappear but the colors get darker:

What happens there is that blue and green get mixed and then probably the browser ignores the sRGB transfer function, which leads to a darker color.

I didn't see such wrong extra stripes with round, and the wrongness of the colors is less severe with round.

Sometimes blue and green are separated (and bright) at the top and mixed (and darker) at the bottom:

And sometimes the other way around.

But you know what else I noticed. Changing only the window size affects these image artifacts (even if I don't pinch-zoom again and thus visualViewport.scale doesn't change again). So maybe this is due to bad rounding when computing the size or coordinates of something (visual viewport, layout viewport, canvas, ...) in CSS pixels or device pixels. For example the coordinates of the visual viewport relative to the layout viewport, or of the canvas relative to the visual viewport.

[greggman]

What happens there is that blue and green get mixed and then probably the browser ignores the sRGB transfer function, which leads to a darker color.

The blue and green get mixed because the device pixel size calculation is incorrect so it's trying to display N pixels in N-1 space or N+1 space and they get blended (when pixelated is not checked). Otherwise, a single line or row of pixels gets doubled or removed when pixelated is checked.

The point is, this is a browser issue. There's nothing a web page can do to get the correct numbers.

I filed a spec bug here:

https://github.com/w3c/csswg-drafts/issues/11412

[page200]

There's nothing a web page can do to get the correct numbers.

Maybe there is. We get a lot of info in the form of digits after the decimal point in various variables that report the scales and coordinates of things. Maybe we're using that info wrong, for example rounding when setting the canvas location and size. For example, left and top of the canvas (its coordinates relative to the layout viewport) should be noninteger in order to "meet" the visual viewport, whose coordinates are a noninteger number of device pixels relative to the layout viewport, at integer device-pixel coordinates. And for left and top to take effect, position shouldn't be static.

Edit: You can update the canvas coordinates to follow the visual viewport around, or better for the user (whose intention was to pinch-zoom into a certain part of the image and to use the browser's capability to rapidly scroll around an unchanging page without the overhead of repainting or repositioning the canvas with JS) make the canvas have left and top larger than 0 but smaller than one pixel. But the canvas shouldn't just stay at coordinate (0, 0) of the layout viewport (or of the page) if that's not an integer (in device pixels) in the coordinate system of the visual viewport.

On a less important note, devicePixelContentBoxSize used in the web.dev article in the context of pixel snapping is not available in Safari.

[greggman]

You could use those factional values. The question though is, if you find something that works, does the spec require them to work or did you just get lucky on a particular device, or a particular browser, or a particular version of a particular browser. If the spec doesn't nail it down (and provide WPT tests), the a browser is free to change its math and break your assumptions.

[page200]

does the spec require them to work

devicePixelContentBox might be considered a somewhat misleading variable name if it requires conversion from the coordinate system of the layout viewport to that of the visual viewport. But I don't know what values that are necessary for this conversion you think are missing in the spec. We have much info such as visualViewport.offsetTop. Safari doesn't follow the spec yet and we probably have a bug in the positioning of the canvas as I described and its sizing (why is the canvas size integer according to your info box and has only the the digits .5 and .25 after the decimal point according to DevTools if visualViewport.scale has lots of digits after the decimal point?) or something, but I'm not aware of the spec not ensuring that we can do whatever we want.

devicePixelContentBox shouldn't get deprecated. Devs might want to use it to prepare something that will look good after the user has pinch-zoomed back out.

[page200]

Possibly related: https://stackoverflow.com/questions/47696956/display-pixel-perfect-canvas-on-all-devices/72611819#72611819

Message ID: @.***>