Lazy loading video element poster attribute

[asuh]

trafficstars

This approach for lazy loading videos is close to perfect.

<video preload="none" src="video.mp4" poster="videoposter.jpg" type="video/mp4">

In a video gallery with many videos, using preload="none" can help speed up page loading with its lazy loading capability. The one missing feature is lazy loading the poster attribute image.

I realize loading="lazy" only works for iframe and img elements, but this one enhancement to the lazy loading capabilities would potentially be a noticeable performance improvement in addition to not preloading the videos.

[annevk]

cc @whatwg/media @domfarolino

[foolip]

Paging some people involved in https://github.com/whatwg/html/pull/3752: @bengreenstein @domenic @jakearchibald @zcorpan

If we just add a loading="lazy" attribute for <video>, should it affect the poster in exactly the same way as <img src>? Or should it also do something for the video resource if no poster attribute is used?

[jakearchibald]

Gut feeling: posterloading="lazy" to control the loading of the poster, and if we want the loading of video to be impacted by viewport intersection, add another option to preload, eg preload="lazymetadata" & preload="lazy".

Otherwise, combinations like loading="lazy" preload="none" are confusing.

[zcorpan]

I wonder if it makes more sense to bring all of img's features to video's poster image by something like

<video use-first-child-img-or-picture-as-poster-also-rename-this-attribute>
 <source ...>
 <picture>
   <source ...>
   <img loading="lazy" decode="async" srcset="..." sizes="...">
 </picture>
</video>

Otherwise how do we add srcset, art direction, and type-based resource selection to poster images?

[zcorpan]

If loading="lazy" appears on a <video> I would expect it to apply to the video, or maybe both the video and the poster. Its behavior is orthogonal to preload in that preload specifies how much to load, and loading specifies when to load (ASAP vs wait until it's in view or almost in view).

[jakearchibald]

I wonder if it makes more sense to bring all of img's features to video's poster image by something like

I like that!

…loading behavior is orthogonal to preload…

loading="lazy" preload="none" seems really weird though

[zcorpan]

It's not so weird if it also applies to poster="...", in my opinion. I guess the loading attribute would have no effect when there's no poster attribute and you use preload="none", sure. Is it a problem? loading="lazy" with src="data:..." is also weird, but isn't disallowed.

[zcorpan]

I think these are the options to solve this issue:

• Add a posterloading=lazy attribute to control lazy-loading of the poster image (but not the video).
• Add a posterelement="#imgid" attribute to point to an <img id=imgid> element to use as the poster image, and support (almost) all of <img>'s features also for poster images. This also solves other feature requests like picture/srcset support for poster images. See #4004
• Don't add any new attribute, but use lazy-loading behavior when preload=none and dimension attributes are specified, as suggested in https://bugs.chromium.org/p/chromium/issues/detail?id=1317849#c6 (personally I don't like this option, as noted in the bug).
• Add a loading=lazy that controls both the poster image and the video with the same attribute.

For lazy-loading of the video:

• Also add loading=lazy to control lazy-loading of the video (but not the poster image).
• Reuse the preload attribute to control lazy-loading of the video (but not the poster image).
• Add a loading=lazy that controls both the poster image and the video with the same attribute. (This is also in the above list.)

[zcorpan]

I'm not sure it's worthwhile to specify lazy-loading for video, since preload="none" exists and people can use lazy-loading poster images (when that is supported).

[scottaohara]

@zcorpan would

I wonder if it makes more sense to bring all of img's features to video's poster image by something like

mean that mean authors would be able to provide alternative text for the poster then? because that'd be something people have wanted for quite some time, and would be most welcome.

[zcorpan]

@scottaohara maybe, but I think it's better to discuss alt text (or accessible name) for video (and audio) in a new issue.

For now, aria-label or aria-labelledby can be used, right?

[scottaohara]

We're talking about different things, @zcorpan. Those attributes can be used to provide a name for the 'video' element, but they would not (and should not) be used to provide a description for the poster itself. This graphic is often used (misused) to provide some sort of title card with content that is only available via that image, and is only relevant when that image is displayed, has long been a frustrating topic for some.

Here is a closed HTML AAM issue which then links to other past issues on the subject. (following the links leads to some pretty heated 'discussion')

Understood if you think this should be spun out into a separate issue, but the reason I brought it up is because of your mention of "all of img's features", and if this idea is accepted, then this image (and thus its alternative text) could be exposed.

[asuh]

Here's what I'm reading from the various conversations above, also with my preference.

<video
  src="url_of_src" // URL of video source
  poster="url_of_img" // URL of image source
  aria-label="Video element name" // Describes the name of the video element, similar to describing a button label
  alt="Description of video content" // Describes the contents of the video itself, similar to image alt text
  loading="lazy" // Lazy load the video element and poster image
  preload="none" // Don't preload the video element, similar to loading="lazy", but could also help define how to lazy load the video
>

It makes more sense for loading="lazy" to be a consistent attribute name with consistent functionality. Even if the loading attribute duplicates most of preload, I would always be inclined to add loading to all media for better performance. Education can help differentiate it from preload and when/where to use either.

[zcorpan]

@scottaohara ok. Yeah, I think a separate issue would be best to align on how to expose the poster image to ATs. It might inform how we want to solve this issue, but still lazy loading and accessibility are orthogonal concerns.

[zcorpan]

I think we should add posterloading=lazy to solve this issue, since it's least complex and the other things can still be added separately (lazy-loading the video with loading=lazy, adding alt text for the poster #7954 could be a separate attribute like posteralt, and srcset/picture support for posters I think should be solved together with #6363).

[zcorpan]

PR: #8428

This issue still has needs implementer interest. @yoavweiss is there interest for Chromium? @annevk for WebKit? @emilio for Gecko?

[annevk]

posterloading seems like a reasonable addition, yes.

[yoavweiss]

^^ @chrishtr for Chromium interest, as his team is likely to own the implementation work.

[chrishtr]

Confirmed interest from Chromium.

[zcorpan]

Thanks @annevk @chrishtr !

What's remaining is review of #8428 and the tests.

[emilio]

This seems fine to me as well. In general it'd be great to have it better-defined how <video poster> works (in Gecko the poster is just an anonymous <img> element), but that might be a bit trickier... It could unlock very cool stuff tho, like something like:

<video ...>
  <picture slot="poster">...</picture>
</video>

or pretty much anything (doesn't have to use the slot attribute, could be magic a la <details>, but you get the idea...).

The shadow tree of <video> could be specified as something like:

#shadow-root
  <slot name=poster></slot>
  <ua-defined-controls></ua-defined-controls>

Where the ua-defined-controls contents are, well, UA-defined, and would include captions, the video controls, etc.

I wouldn't oppose to the one-off extra attribute, but that would be perhaps a more flexible / better design over-all... Would there be appetite for such a thing?

[zcorpan]

Thanks @emilio

That would be like the second option in https://github.com/whatwg/html/issues/6636#issuecomment-1125404102 but without ID lookup (instead relying on picture or img being a child or descendant of video + some opt-in).

It's more work, but we get more features. Curious to hear what others think.

[johannesodland]

I wonder how this would look if the source media attribute was reintroduced later as in #6363.

[scottaohara]

It's more work, but we get more features. Curious to hear what others think.

just chiming in again that the idea of using picture > img within a video would be welcome for providing alternative text for the poster image. https://github.com/whatwg/html/issues/7954

[annevk]

How common is poster? A bigger redesign might be worth doing if it's popular and also if there are already popular libraries available that address some of the shortcomings.

[zcorpan]

Web Almanac: https://docs.google.com/spreadsheets/d/1T5oVAVmcH3sM6R-WwH4ksr2jFtPhuLXs3-iXXoABb3E/edit#gid=1515756004 - though poster appears multiple times and I'm not sure how to interpret pct_attribute and pct_videos. @eeeps can you help?

Chromium use counter V8HTMLVideoElement_Poster_AttributeGetter is at 2.1123% and V8HTMLVideoElement_Poster_AttributeSetter at 2.0306%, but I assume this doesn't count use in HTML without author JS accessing the getter/setter, and if so, usage is probably higher.

A quick query in httparchive sample_data (10,000 pages) gives 445 pages = 4.45% of pages, and 1743 pages use video, so 25,5% of pages with a video element also use poster.

queries

SELECT COUNT(DISTINCT page) AS num FROM `httparchive.sample_data.response_bodies_desktop_10k` WHERE
REGEXP_CONTAINS(body, r'(?i)(<video\s+[^>]*\sposter\s*=)')

445

SELECT COUNT(DISTINCT page) AS num FROM `httparchive.sample_data.response_bodies_desktop_10k` WHERE
REGEXP_CONTAINS(body, r'(?i)(<video\s+)')

1743

[kevinfarrugia]

Incidentally I was looking at a similar query and the results are similar to what you have in the sample data. Uses the same data set as the Web Almanac 2022, (June 2022) sampled at 10%.

+---------+-------------------+-------------------+-----------------------+-------------+
| client  | pages_with_videos | pages_with_poster | pct_pages_with_poster | total_pages |
+---------+-------------------+-------------------+-----------------------+-------------+
| mobile  |             38847 |             10458 |    0.2692099776044482 |      771291 |
| desktop |             35098 |             10071 |   0.28693942674796286 |      558537 |
+---------+-------------------+-------------------+-----------------------+-------------+

pct_pages_with_poster only includes pages which have at least one video element.

Query

#standardSQL

CREATE TEMPORARY FUNCTION getAttributes(payload STRING)
RETURNS STRUCT<poster INT64> LANGUAGE js AS '''
try {
  var almanac = JSON.parse(payload);

  var obj = {
    poster: 0,
  };

  for (var node of almanac.videos.nodes) {
    if (typeof node.poster != "undefined") {
      obj.poster++;
    }
  }

  return obj;
} catch (e) {
  return {};
}
''';

SELECT
  client,
  COUNTIF(total_video > 0) AS pages_with_videos,
  COUNTIF(attributes.poster > 0) AS pages_with_poster,
  COUNTIF(attributes.poster > 0) / COUNTIF(total_video > 0) AS pct_pages_with_poster,
  COUNT(0) AS total_pages
FROM (
  SELECT
    _TABLE_SUFFIX AS client,
    SAFE_CAST(JSON_EXTRACT_SCALAR(JSON_EXTRACT_SCALAR(payload, '$._almanac'), '$.videos.total') AS INT64) AS total_video,
    getAttributes(JSON_EXTRACT_SCALAR(payload, '$._almanac')) AS attributes,
  FROM
    `httparchive.pages.2022_06_01_*` TABLESAMPLE SYSTEM(10 PERCENT)
)
GROUP BY
  client

[zcorpan]

I discussed with @chrishtr, he said supporting picture might be too much complexity. But supporting img still gives srcset, alt, and other features of img.

@emilio wrote

<video ...>
  <picture slot="poster">...</picture>
</video>

or pretty much anything (doesn't have to use the slot attribute, could be magic a la <details>, but you get the idea...).

Yeah, I think we shouldn't use slot for an HTML-native feature. We could use a new boolean attribute on video:

<video posterchild>
  <img ...>
  <source ...>
</video>

(Please suggest other names for posterchild, although I also kinda like it 🙂.)

[annevk]

Does <video> need an attribute? Is an <img> child not sufficient?

[johannesodland]

I discussed with @chrishtr, he said supporting picture might be too much complexity. But supporting img still gives srcset, alt, and other features of img.

Given the resent reintroduction of the media attribute on media element sources, do you think it's worthwhile to reconsider the picture element? I'm concerned there might be an inconsistency if the video element is able to support art-direction for videos but not for the poster image.