html Add sizes=auto to lazy-loaded <img>

This allows web developers to omit the sizes attribute, or explicitly specify sizes=auto, for <img> elements that have loading=lazy. The browser will use the layout width of the image as input to the srcset resource selection (lazy images don't start loading until layout is known, at the earliest). For eager-loading images, it's invalid and equivalent to 100vw.

Fixes #4654.

[ ] At least two implementers are interested (and none opposed):
- Chromium: yes https://github.com/whatwg/html/issues/4654#issuecomment-1154412434
- Gecko: ? https://github.com/mozilla/standards-positions/issues/650
- WebKit: ? https://github.com/WebKit/standards-positions/issues/9
[x] Tests are written and can be reviewed and commented upon at:
- https://github.com/web-platform-tests/wpt/pull/34427
[ ] Implementation bugs are filed:
- Chrome: …
- Firefox: …
- Safari: …
- Deno (only for timers, structured clone, base64 utils, channel messaging, module resolution, web workers, and web storage): N/A
- Node.js (only for timers, structured clone, base64 utils, channel messaging, and module resolution): N/A
- Validator.nu: https://github.com/validator/validator/issues/1396

(See WHATWG Working Mode: Changes for more details.)

:boom: Error: Wattsi server error :boom:

PR Preview failed to build. (Last tried on Aug 31, 2022, 3:24 PM UTC).

More

PR Preview relies on a number of web services to run. There seems to be an issue with the following one:

:rotating_light: Wattsi Server - Wattsi Server is the web service used to build the WHATWG HTML spec.

:link: Related URL

Parsing MDN data...
Parsing...
Generating HTML variant...
Error: missing <dfn> for topic "natural dimensions" explicitly from <span> element containing "natural dimensions"; previous heading contents are "4.8.4.2.2 Sizes attributes"
Error count: 1
Saving index-html
Splitting...
Saving index.html
Saving introduction.html
Saving infrastructure.html
Saving common-microsyntaxes.html
Saving urls-and-fetching.html
Saving common-dom-interfaces.html
Saving structured-data.html
Saving dom.html
Saving semantics.html
Saving sections.html
Saving grouping-content.html
Saving text-level-semantics.html
Saving links.html
Saving edits.html
Saving embedded-content.html
Saving images.html
Saving iframe-embed-object.html
Saving media.html
Saving image-maps.html
Saving embedded-content-other.html
Saving tables.html
Saving forms.html
Saving input.html
Saving form-elements.html
Saving form-control-infrastructure.html
Saving interactive-elements.html
Saving scripting.html
Saving canvas.html
Saving custom-elements.html
Saving semantics-other.html
Saving microdata.html
Saving interaction.html
Saving dnd.html
Saving browsers.html
Saving window-object.html
Saving origin.html
Saving history.html
Saving browsing-the-web.html
Saving webappapis.html
Saving dynamic-markup-insertion.html
Saving timers-and-user-prompts.html
Saving system-state.html
Saving imagebitmap-and-animations.html
Saving comms.html
Saving server-sent-events.html
Saving web-messaging.html
Saving workers.html
Saving worklets.html
Saving webstorage.html
Saving syntax.html
Saving parsing.html
Saving named-characters.html
Saving xhtml.html
Saving rendering.html
Saving obsolete.html
Saving iana.html
Saving indices.html
Saving references.html
Saving acknowledgements.html
Generating DEV variant...
Splitting...
Saving index.html
Saving introduction.html
Saving infrastructure.html
Saving common-microsyntaxes.html
Saving urls-and-fetching.html
Saving common-dom-interfaces.html
Saving structured-data.html
Saving dom.html
Saving semantics.html
Saving sections.html
Saving grouping-content.html
Saving text-level-semantics.html
Saving links.html
Saving edits.html
Saving embedded-content.html
Saving images.html
Saving iframe-embed-object.html
Saving media.html
Saving image-maps.html
Saving embedded-content-other.html
Saving tables.html
Saving forms.html
Saving input.html
Saving form-elements.html
Saving form-control-infrastructure.html
Saving interactive-elements.html
Saving scripting.html
Saving canvas.html
Saving custom-elements.html
Saving semantics-other.html
Saving microdata.html
Saving interaction.html
Saving dnd.html
Saving browsers.html
Saving window-object.html
Saving origin.html
Saving history.html
Saving browsing-the-web.html
Saving webappapis.html
Saving dynamic-markup-insertion.html
Saving timers-and-user-prompts.html
Saving system-state.html
Saving imagebitmap-and-animations.html
Saving comms.html
Saving server-sent-events.html
Saving web-messaging.html
Saving workers.html
Saving worklets.html
Saving webstorage.html
Saving syntax.html
Saving named-characters.html
Saving xhtml.html
Saving obsolete.html
Saving indices.html
Saving references.html
Saving acknowledgements.html
Generating SNAP variant...
Error: missing <dfn> for topic "natural dimensions" explicitly from <span> element containing "natural dimensions"; previous heading contents are "4.8.4.2.2 Sizes attributes"
Error count: 1
Saving index-snap

If you don't have enough information above to solve the error by yourself (or to understand to which web service the error is related to, if any), please file an issue.

Jun 14 '22 00:06 zcorpan

I wonder if the feature should more explicitly check for the case of no dimensions being specified, since the behavior is a bit weird in that case.

When no image is loaded yet, the width will be 0px, in which case the sizes algorithm returns 100vw.
When an image is loaded, and a relevant mutation occurs (for example, the viewport width changes, making the rendered width of the image change because the used sizes was 100vw, and this is now a relevant mutation). This time, the parsed sizes will return the current rendered width of the image, in pixels. So it's no longer responsive to further changes of the viewport width.

So, how to fix this? Use 100vw if the computed value (not used value) of the 'width' and 'min-width' properties are both auto?

Jul 06 '22 13:07 zcorpan

So, how to fix this? Use 100vw if the computed value (not used value) of the 'width' and 'min-width' properties are both auto?

Using 100vw when we don't have any better alternatives (since we don't know what the layout size would be), SGTM. The "computed-value width isauto" case sounds like it fits that description. I'm not sure if there aren't other cases (but you probably know).

Jul 08 '22 09:07 yoavweiss

So, how to fix this? Use 100vw if the computed value (not used value) of the 'width' and 'min-width' properties are both auto?

Done in fc651c8993932014a8ab67cd27e2087ebff9a37e.

I suppose since the 'aspect-ratio' property now exists, it's possible to specify a width by setting a height and an aspect-ratio. So if width and min-width are both auto, we should check if 'aspect-ratio' and 'height'/'min-height' are also auto.

Are there any other ways to specify a width? cc @whatwg/css

Aug 08 '22 13:08 zcorpan

To answer my own question, I think we have these:

'inline-size' and 'block-size'; which is the width depends on 'writing-mode'
sizing values (css-sizing-3 and css-sizing-4) can also be 'none', 'min-content', 'max-content', 'fit-content(...)', 'stretch', 'fit-content', 'contain', which in some cases act like 'auto'. Maybe we should check if the computed value is a <length-percentage> instead of checking that it's not 'auto'.
- var() is substituted at computed-value time, so should work ok
- not sure what to make of math functions
as far as I can tell, per CSS2, for replaced elements (which img is before the image has loaded), the 'left' and 'right' properties can't be used to specify a width.

So, an img element has a "specified width" if at least one of the following is true:

the computed value of 'width' or 'min-width' is a <length-percentage>
the computed value of 'aspect-ratio' is not 'auto' and the computed value of 'height' or 'min-height' is a <length-percentage>
the computed value of 'writing-mode' is 'horizontal-tb' and
- the computed value of 'inline-size' or 'min-inline-size' is a <lenght-percentage> or
- the computed value of 'aspect-ratio' is not 'auto' and is not degenerate and the computed value of 'block-size' or 'min-block-size' is a <length-percentage>
the computed value of 'writing-mode' is not 'horizontal-tb' and
- the computed value of 'block-size' or 'min-block-size' is a <lenght-percentage> or
- the computed value of 'aspect-ratio' is not 'auto' and is not degenerate and the computed value of 'inline-size' or 'min-inline-size' is a <length-percentage>

Does that seem correct?

Edit: I now also found these properties: https://drafts.csswg.org/css-sizing-4/#intrinsic-size-override

Aug 09 '22 10:08 zcorpan

Maybe a more general approach would be:

An element's concrete object size ignoring natural dimensions is its concrete object size but acting as if there are no natural dimensions, thus producing a rectangle with an absolute width and height.

This will match the actual concrete object size before the image has loaded.

Then, we can check that the "concrete object size ignoring natural dimensions" width is > 0, and any changes to this width is a relevant mutation.

Aug 09 '22 12:08 zcorpan

I went with the approach described in the comment above. I believe it solves the problem discussed in https://github.com/whatwg/html/pull/8008#issuecomment-1176203523 and, at least in spec terms, doesn't introduce a lot of complexity.

Aug 09 '22 13:08 zcorpan

(Need to rebase this after https://github.com/whatwg/html/pull/8175 is merged.)

Aug 09 '22 14:08 zcorpan

@yoavweiss PTAL :)

Aug 11 '22 10:08 zcorpan

I'm a bit worried about compat for making more things "relevant mutations". Should we gate the new additions on usage of w descriptors in srcset?

Aug 11 '22 16:08 zcorpan

I'm a bit worried about compat for making more things "relevant mutations". Should we gate the new additions on usage of w descriptors in srcset?

I share that concern, and not sure that gating it would help significantly. I understand the motivations for loading attribute mutation changes, but unclear on mutations when the concrete element size changes.

Aug 12 '22 12:08 yoavweiss

The size observer is to keep the image "responsive" to changes in rendered size. For example, consider a gallery with thumbnails and clicking a thumbnail shows the same img as large as possible within the viewport. The environment hasn't changed, only the rendered width. It seems reasonable to assume that sizes=auto would not make the initial size "sticky".

Aug 15 '22 17:08 zcorpan

OK, I understand the reasoning to include the concrete size in the relevant mutations, and I think it makes sense to restrict those to only cases where they matter (so when srcset has 'w' descriptors). Can you run an HTTPArchive analysis to see how often that condition is met?

IIUC, such content currently would be loaded once with a size of 100vw, and once we switch the the proposed behavior, multiple images may be loaded, if the image size is significantly increased or decreased. Is that understanding correct?

Aug 31 '22 12:08 yoavweiss

OK, I understand the reasoning to include the concrete size in the relevant mutations, and I think it makes sense to restrict those to only cases where they matter (so when srcset has 'w' descriptors).

Changed in 527e628630adf3bb20fb83e01e72e15c9b7ce3db

Can you run an HTTPArchive analysis to see how often that condition is met?

Do you mean how many pages use 'w' descriptors and no sizes attribute?

IIUC, such content currently would be loaded once with a size of 100vw, and once we switch the the proposed behavior, multiple images may be loaded, if the image size is significantly increased or decreased. Is that understanding correct?

Yes, or it might not change which image is loaded but still fire a load or error event in response to changing from display: none to display: inline or changing the rendered width. But now this only happens for pages that use w descriptors (which is ~8.2% of page views).

Aug 31 '22 15:08 zcorpan

Do you mean how many pages use 'w' descriptors and no sizes attribute?

'w' descriptors, no sizes attribute and are currently lazy loaded.

Sep 01 '22 07:09 yoavweiss

Yes, or it might not change which image is loaded but still fire a load or error event in response to changing from display: none to display: inline or changing the rendered width. But now this only happens for pages that use w descriptors (which is ~8.2% of page views).

Yeah, the load/error event bits are scary, as it can easily break developer expectations. HA search can be a good first step. I can also add Chromium use counters..

Sep 01 '22 08:09 yoavweiss

OK I tried to query for that through a series of queries, and ended up with a result of 51,997 pages out of 15,069,836 total pages (so 0.345%). Of those, 4360 use data-sizes="auto".

https://docs.google.com/spreadsheets/d/1pquIGynR0RQj7CO4pHKw2E1Pzdby94Z-vfPToXoYTnA/edit?usp=sharing (this is a subset because the result was too large to export).

Sep 05 '22 15:09 zcorpan

That's a lot, but initial (and not at all stable) data from Canary suggests it may even be higher when looking at page visits.. :/

Sep 07 '22 13:09 yoavweiss

A lot of difference between different platforms, so we probably want to let the use counter bake a bit before drawing conclusions.

Sep 07 '22 13:09 yoavweiss

How long should it bake?

Sep 29 '22 18:09 zcorpan

Seems like initial data has reached the dashboards, and it's at 1.2%. I'm seeing significantly less than that in internal dashboards, so it might be best to wait for the data to stabilize before jumping to conclusions.

Ideally, we'd want a week's worth of data from Stable, which we'll likely have early November.

Sep 30 '22 04:09 yoavweiss

It's not November yet, but the current value is 1.95%.

Oct 28 '22 12:10 zcorpan

If this change appears too risky, I think we could first try to change the “load the image data” algorithm to avoid firing “load” and “error” events in more cases. If that is compatible, then it should alleviate the main compat risk for auto-sizes.

Oct 28 '22 12:10 zcorpan

The monthly average for November 1 is 2.49%.

Nov 09 '22 14:11 zcorpan

Apologies for my delay here. After discussion with @progers, it turned out that my initial usecounter wasn't correct. I added a new one, which is currently at 0.001% in Canary/Beta. From the previous counter, I think we can conclude that there aren't significant differences between the proportions we see in Beta vs. Stable, so I think we can conclude that this change is safe.

At the same time, I kinda like the option of avoiding firing the "load"/"error" events in cases where the image didn't actually loaded.

Nov 10 '22 12:11 yoavweiss

Thanks @yoavweiss! Nice to see better numbers with the corrected use counter.

We still need interest from Gecko or WebKit. @emilio @hober any update?

At the same time, I kinda like the option of avoiding firing the "load"/"error" events in cases where the image didn't actually loaded.

Yeah. Let's track this in a separate issue: #8492

Nov 10 '22 14:11 zcorpan

Any chance sizes=auto could also be supported for JavaScript-inserted images? (Or is the implementation model there too different?)

This should benefit

React-rendered static sites (e.g., Framer sites are written in React; they load as server-rendered HTML during the first visit but use React for subsequent navigations)
apps without server-side rendering
etc

The idea is: given that browsers already have CSSOM built by the time the image is inserted into DOM, they could automatically pick the right image size to load.

Dec 11 '22 20:12 iamakulov

@iamakulov as specified it would apply equally to JS-inserted images as for parser-inserted images.

The idea is: given that browsers already have CSSOM built by the time the image is inserted into DOM,

This is not correct. JS can run and insert images before the first layout. Example:

<!doctype html>
<script src=a.js async></script>
<link rel=stylesheet href=b.css></link>
...

If a.js is faster to load and execute than it takes for b.css to load, it will run before the first layout.

Dec 13 '22 12:12 zcorpan

Updated OP for implementation interest and filed bugs. This now needs a rebase though.

Mar 01 '23 10:03 zcorpan

I've rebased. The build fails because of https://github.com/whatwg/html/pull/8175

Mar 08 '23 13:03 zcorpan

@zcorpan - Prior to doing the load, does any layout information change with this PR?

Specifically today (cross-browser), an image like: <img src="loading.png"> will have a zero natural-size / intrinsic-size (depending on what terminology you use), is this still the case?

(note this will mean that <img src="loading.png" style="max-width: 10px;"> will have a size of 0px during loading).

Apr 26 '23 21:04 bfgeek