Make `<embed>` and `<object>` behave like `<iframe>` for image loading

[farre]

I'd like to propose that we change how <embed> and <object> load resources with a type that starts with image/ to be like that what we have for media resources (e.g. <video>), and behave like "7.11.6 Page load processing model for media". This essentially means removing the special case for images in the <embed> and <object> setup steps, and will make <embed> and <object> behave more like <iframe> when loading images.

Making <embed> and <object> behave more like

[domenic]

I am extremely enthusiastic about this, if you think we can get away with it compatibly.

The other thing this probably breaks (unless you add back in some different complications) is fallback for <object>; see https://github.com/whatwg/html/issues/7327 for discussion there, including some use counter data. I guess <embed> also has special display for the error case, but I'm not sure if that's web-observable.

[domenic]

Hmm I think I misread your proposal as "make embed and object behave as browsing context containers like iframe that get navigated to the target URL", not "continue having their current setup but remove the image special case".

In particular the current setup does not use the usual navigation algorithm, but instead uses a separate path to get the response, and then process that response as either an image, bad-response, or nested browsing context case. I guess you are suggesting sticking with the separate get-response and process-response stages, and just having process-response have only bad-response or nested browsing context cases.

That could work, and does remove some of the risk of my full-unification plan. However the remaining issue (and also something we're not sure yet how to handle in #4592) is that <object>'s special sizing behavior for images is not something iframes support. iframes always have 300x150 size, unless you supply width=""/height="" attributes. Do you think we'd have the same behavior for <object>, or would we add some kind of hack to resize the <object> once the response comes back if it's an image, even though that image is going into a nested browsing context?

[farre]

Exactly, only bad-response or that there will be a nested browsing context.

I'm looking at this for Firefox in the context of ORB, since nesting an image load in a synthetic document would nicely separate a cross-origin <object> embedded image from its embedder. I've found exactly the sizing issue, but also three other issues:

1. There is at least one style that needs to be propagated to the wrapped image, namely image-rendering. I haven't found more, but that doesn't mean that there aren't any, and this might need further investigation.
2. The fact that if embedding images result in a nested browsing context it also shows up in its parents Window.frames, but also:
3. This means that embedded images can now be name targeted when navigating.

Implementation-wise, I've managed to perform the hack you describe to let the element know about the intrinsic size of the image, and I've also been able to implement propagation of image-rendering.

The thing that's stumped me is what to do about Windows.frames and named targeting, which is why I wanted to know if it would be possible to adjust the spec to allow this.

This is only my opinion, but I'd much rather have <object> and <embed> have 300x150 size unless otherwise specified as well as allowing embedded image containers to expose their browsing context's functionality as well. This would make images work like other media, e.g. video/webm, at least if you consider how both Firefox and Chrome implement embedding of media resources in <object>.

What to do about styling I'm not that sure of though.

[domenic]

In general I'd be happy to adjust the spec to anything web-compatible that moves embed/object closer to iframe. That includes window.frames, named targeting, and sizing. Figuring out the compat burden is the hard part though.

[domenic]

@mfreed7 maybe you or your team could help determine the compat impact of removing the auto-sizing, by adding a use counter?

My understanding based on some quick tests is that <embed> and <object> both get some sort of intrinsic size set, when they host a SVG. I think it is only when they host a SVG, but I am not sure. See http://software.hixie.ch/utilities/js/live-dom-viewer/?saved=10404.

This is an information leak from the hosted content to the outer page that we would like to remove. And it also makes embed/object extra-complicated. And it gives embed/object extra capabilities over iframe, which makes it harder to tell people to stop using embed/object.

Could we add a use counter to tell in what situations this intrinsic size information affects the outer page? E.g. calculate the size without it (which would normally be 300x150, or whatever the width="" and/or height="" attributes override those to); calculate the sizes with this intrinsic size info; and then see on what percentage of pages that mismatches.

[mfreed7]

We can probably tackle that at some point soon. It might be a tricky thing to use count, since it might require doing extra layout work. But we can take a look.

Tracking bug here: https://crbug.com/1340033

[zcorpan]

For reference, this was fixed in Gecko in https://bugzilla.mozilla.org/show_bug.cgi?id=1595491

[domenic]

Wow, that's amazing! Including removing auto-sizing??

[zcorpan]

No, auto-sizing still happens. We also don't update window.length or allow the object to be targeted by links.

Testing with https://software.hixie.ch/utilities/js/live-dom-viewer/saved/14032 it seems WebKit and Chromium now create a normal browsing context, 300x150, and allow link targeting.

So we can simplify in Gecko. @farre worked hard to make auto-sizing work and prevent link targeting, but it is what it is. :)

[domenic]

Testing with https://software.hixie.ch/utilities/js/live-dom-viewer/saved/14032 it seems WebKit and Chromium now create a normal browsing context, 300x150, and allow link targeting.

It is very surprising that this changed without any of us noticing... I wonder if we're testing right? Is the auto-sizing only for SVG or something? I'll ask around to see this just happened without us noticing...

[mfreed7]

Testing with https://software.hixie.ch/utilities/js/live-dom-viewer/saved/14032 it seems WebKit and Chromium now create a normal browsing context, 300x150, and allow link targeting.

I'm not sure what time period the "now" in that sentence refers to, but I just tried to bisect this, and 300x150 goes back at least to Chrome 90.

[domenic]

I think auto-sizing in Chromium might be dependent on the file extension in the URL... another reason these elements are terrible.

• https://software.hixie.ch/utilities/js/live-dom-viewer/?saved=14039
• https://software.hixie.ch/utilities/js/live-dom-viewer/?saved=14040
• https://software.hixie.ch/utilities/js/live-dom-viewer/?saved=14041

I believe the code path is this layout-adjacent code which bottoms out in this MIME type from URL code.

[zcorpan]

Hmmm, I don't like including file extensions from the system registry...

https://software.hixie.ch/utilities/js/live-dom-viewer/saved/14043 shows which image file extensions get the "image" treatment in Chromium and WebKit. (The list of extensions comes from ChatGPT.)

For me on macOS:

Chrome Canary: ico, nii.gz, apng, gif, bmp, png, tif, tiff, xbm, avif, jfif, jpeg, jpg, webp, eps, ps Safari TP: 3fr, arw, cr2, cr3, crw, dcr, dng, erf, fff, iiq, mos, mrw, nef, nrw, orf, pef, raf, raw, rw2, rwl, sr2, srf, srw, cpt, kra, ora, psd, img, ico, dcm, nii.gz, pic, gif, bmp, dib, pbm, pcx, pgm, png, pnm, ppm, ras, rgb, sgi, tga, tif, tiff, xbm, xpm, xwd, avif, heic, heics, heif, heifs, hif, j2k, jp2, jpe, jpeg, jpf, jpg, jpx, jxl, webp, ai, eps, pdf, ps

I'll take a look at httparchive also.

[zcorpan]

It seems .swf also doesn't get a browsing context, though. Is the not-supported-plugin path a hardcoded list of extensions?

[foolip]

This was discussed in https://github.com/whatwg/html/issues/11622 today. There's more httparchive research to be done to understand the compat risk, and @zcorpan has an export of httparchive results to look at now.

[zcorpan]

Thanks @foolip !

Total number of pages is 22,624,599. The first query finds top-level pages that have an object or embed element.

There are 6688 pages using object or embed without width, height, type attributes and a file extension in the URL.

https://docs.google.com/spreadsheets/d/1wuqST3xcvXwonuGQaU2v7KN8g3lWd_nGoy-Y-ckFAtA/edit?usp=sharing

The most common extensions:

file_ext	page_count
png	1691
svg	1551
jpg	1145
swf	980
js	757
mp3	348
pdf	311
php	194
html	148
webp	103
mp4	101
mid	50
jpeg	44
wav	42
gif	41

https://docs.google.com/spreadsheets/d/1IwHxkn7ZqGwS5cRwVSUVaAuBeoQzOtjfsrnMeHiw44M/edit?usp=sharing

Filtering down to only those that have a file extension that is supported (or doesn't create a nested browsing context) in Safari TP (see above), there are 2466 pages (~0.01% of all pages):

https://docs.google.com/spreadsheets/d/148LSdyi70vN12uo2iFlxQM_4MsPvr34KAKJpexyStHE/edit?usp=sharing

[zcorpan]

I guess I shouldn't have excluded width and height to find all pages that use file-extension based type.

Things to find out:

• Do we need to pay attention to the file extension? If yes, which extensions are needed for compat.
• Do we need auto-sizing?
• Do we need to hide the nested browsing context?

These will need somewhat different queries.

Maybe use counters could help, especially for the third one. e.g.

• object or embed goes through the "image" code path and type was detected from the file extension.
• object or embed goes through the "image" code path and does not have both specified width and height (attributes or CSS).
• object or embed goes through the "image" code path and the page accesses a nested browsing context by indexing.

[zcorpan]

Gecko waits with deciding whether to create a nested browsing context until there is a response.

Since waiting for a response is needed to decide whether to show fallback, I think this model is better than what Chromium and WebKit are doing. It also means there's no need to check for file extensions. Per @farre, waiting for the response is essential because if we're doing a cross-origin image load, the synthetic document also needs to be cross-origin (and so needs a process switch).

Demo for fallback: https://software.hixie.ch/utilities/js/live-dom-viewer/saved/14046

• Gecko doesn't create a nested browsing context.
• Chromium creates a nested browsing context but then destroys it when showing fallback.
• WebKit creates a nested browsing context and keeps it around when showing fallback.

[cwilso]

I believe this is waiting on more httparchive research, and can have agenda+ removed?

[evilpie]

As @zcorpan said, Firefox doesn't use the file extension for deciding whether to load something as an image. But we do use the type attribute. In fact it seems like all browsers are now aligned and only use the type attribute when loading images, or show an early fallback when the type attribute contains something that is not renderable. Otherwise for renderable types, the type attribute is just ignored and especially doesn't force something to load as text/html which might not otherwise load that way.