Visual tool to scan a page and report missing width/height elements

[adamsilverstein]

This task was suggested by Aymen Loukil for the image focus, however it may fit better in the measurement focus.

[eclarke1]

@adamsilverstein I'll add this to the measurement focus for now and into the backlog, but let me know if that should change.

[dainemawer]

@eclarke1 @adamsilverstein I think I could take a stab at this - sounds like a fun feature to build out.

[eclarke1]

Thanks @dainemawer we can assign this issue to you then and it would be great to move over to 'In Progress' once this is started please

[dainemawer]

@adamsilverstein before I move onto this ticket - do you have any thoughts with regards to implementation? Is this something that we would invision living in WordPress, or separate, like a chrome extension?

[westonruter]

Just to note that sometimes authors intentionally omit width/height attributes such as when adding an remote image that changes randomly. See https://github.com/WordPress/performance/issues/49#issuecomment-995343996. Surely not the normal case, but I just wanted to mention the use case.

[dainemawer]

@felixarntz @ThierryA @adamsilverstein @AymenLoukil - I have a minor proof of concept ready here, but I just want to make sure that this is something we actively want to pursue, considering where we currently are with images overall? This feature is blurring the lines between what Lighthouse does pretty well already in terms of flagging images, so I guess my question is - should we rely on Lighthouse and will users know to use it? Or should we looking into incorporating this tool into the plugin?

[swissspidy]

@westonruter Just stumbled upon this — maybe a candidate for Optimization Detective?

[westonruter]

@swissspidy Hummm. Yes! Beyond just reporting issues with missing width and height on elements, it can measure the dimensions of those elements and supply the missing width and height attributes during optimization. 🎉

[westonruter]

In the case where an image lacks width/height due to the loaded image having variable dimensions, it will be important that we collect multiple URL metrics for a URL before attempting to supply missing width and height attributes. Only once we have multiple URL metrics and the element ends up with the same width/height across each page load can we safely supply the dimensions.

[westonruter]

As noted in https://github.com/WordPress/performance/pull/1420:

I think detect.js may need to be extend to obtain the intrinsic dimensions of elements that have them: img, video, canvas, svg (maybe), and input[type=image] (ewwww).

So for an image, I believe this would be by getting the naturalWidth & naturalHeight, and for video it would be the videoWidth and videoHeight properties.

With these stored in URL metrics, the optimization pass would then set the missing width and height attributes to match. Another custom attribute should also be added like data-od-using-intrinsic-dimensions so that the detection logic knows it should continue to obtain the intrinsic dimensions for storage, if it isn't doing so already.

[westonruter]

Related to this, I just learned that VIDEO tags in the Video block are not inserted with width and height attributes, which means they can cause layout shift. See https://github.com/WordPress/gutenberg/issues/52185.

[westonruter]

Here's an example portrait video on a 3G connection without preload specified and no poster:

https://github.com/user-attachments/assets/bfe51e33-7614-4229-b1fd-1de75a515635

<video controls="" muted="" src="http://localhost:8888/wp-content/uploads/2024/10/goat-18139442-hd_1080_1920_30fps.mp4"></video>

And with a poster supplied, in which case the poster image dimensions become the video's initial intrinsic dimensions:

https://github.com/user-attachments/assets/1b73af61-d7bc-495f-acc1-416c78c19938

<video controls="" muted="" poster="http://localhost:8888/wp-content/uploads/2024/10/goat-poster-jpg.webp" src="http://localhost:8888/wp-content/uploads/2024/10/goat-18139442-hd_1080_1920_30fps.mp4"></video>

And without poster and with preload=none:

https://github.com/user-attachments/assets/ee47da09-52bd-4dcd-b98e-99ad43c34bdb

<video controls="" muted="" preload="none" src="http://localhost:8888/wp-content/uploads/2024/10/goat-18139442-hd_1080_1920_30fps.mp4"></video>

Note what happens when a landscape image is set as the poster:

https://github.com/user-attachments/assets/a094faca-bdaa-4665-b404-83f8868674c0

All of these layout shifts would be eliminated if we supplied the width and height on the VIDEO to correspond to the videoWidth and videoHeight as discovered client-side.

I've also filed https://github.com/WordPress/performance/issues/1592 for setting appropriate preload values based on whether a video is in the initial viewport. (Note: I initially commented here in error that the default preload value is auto when in reality it is up to the browser, but the spec encourages metadata as the default.)

[westonruter]

I'm exploring this in the context of a separate repo at the moment, although I intend to open a PR to implement it as part of Image Prioritizer once it is stable: https://github.com/westonruter/od-intrinsic-dimensions

[westonruter]

Here's an example of the impact of this prototype Intrinsic Dimensions plugin when used with a Video block, here specifically a portrait video which did not have a poster image supplied.

Before	After

As you can see, the elimination of the layout shift is a dramatic improvement in the page experience. It reduces CLS from poor to good:

CLS Before	CLS After

[westonruter]

My assumption was that WordPress wasn't able to determine the dimensions of video files, but I was wrong. It turns out that wp_generate_attachment_metadata() has explicit support for obtaining video metadata via wp_read_video_metadata(). So this makes the use of Optimization Detective to measure the intrinsic dimensions on frontend much less important, since videos uploaded to WordPress should have their dimensions available. We just need to update the Video block in Gutenberg to supply the dimensions and the aspect-ratio, as implemented in the tag visitor:

// Set the width and height to reflect the captured intrinsic dimensions.
$processor->set_attribute( 'width', (string) $intrinsic_dimensions['width'] );
$processor->set_attribute( 'height', (string) $intrinsic_dimensions['height'] );
if ( 'VIDEO' === $processor->get_tag() ) {
	// TODO: It's not clear why the aspect-ratio needs to be specified when the user agent style is already defining `aspect-ratio: auto $width / $height;`.
	// TODO: Also, the Video block has styles the VIDEO with width:100% but it lacks height:auto. Why?
	// TODO: Why does the Image block use width:content-fit?
	$style = sprintf( 'height: auto; width: 100%%; aspect-ratio: %d / %d;', $intrinsic_dimensions['width'], $intrinsic_dimensions['height'] );

	$old_style = $processor->get_attribute( 'style' );
	if ( is_string( $old_style ) ) {
		$style .= $old_style;
	}

	$processor->set_attribute( 'style', $style );
}

Where Optimization Detective is helpful is obtaining dimensions for videos (and images) which are loaded externally.

[swissspidy]

Not sure how reliable wp_read_video_metadata() really is as it relies on metadata. Might not always be available.

[westonruter]

I've opened https://github.com/WordPress/gutenberg/pull/70293 to supply the width and height to the video tag in the Video block when the metadata is available. The changes in this PR are also available in a standalone plugin for existing sites to use before the change is published in a Gutenberg release: Layout-stabilized Video Block.

[westonruter]

I blogged a writeup about Eliminating Layout Shifts in the Video Block.