AccessKit Disable GIFs: Implement poster-based and css-based pausing

[marcustyphoon]

Description

This implements, depending on how you count, two or three new techniques to pause animated images. This yields:

• proper image pausing of elements we previously replaced with a flat color
• faster image pausing
• support for pausing webp images if and only if they're animated

This enables future PRs to implement:

• pausing animated images in many more places, including nearly every animated image anywhere on tumblr (#1729; one may simply consider that PR the canonical version of these changes)
• adding an option to download most animated images only when they're hovered, saving bandwidth and increasing endless scrolling load performance on slower networks (#1728)

Supersedes #1681 and #1709 (they are effectively combined, for non-poster GIFs) and #1707 (included, for with-poster GIFs).

Technical Details

Images with poster elements are paused by showing the poster element and hiding the image. Other elements are paused with CSS overrides (content for images, background-image for elements with it set to a gif) set to a blob URL created by fetching the image source and pausing it.

Non-animated images are ignored (unless they have posters, or have an animation flag but only have one frame, or the Firefox version is between 121 and 132).

details

Currently, AccessKit pauses animated images by waiting until they load, and inserting canvas elements with paused versions of their contents into the DOM that cover them when they are not hovered. It "pauses" elements with animated images as part of their background-image property by replacing the background image with a solid color.

• When this code finds an animated* image beneath the "poster" element provided by Tumblr on GIFs in posts, it applies CSS that shows the poster instead of the image unless the stack is hovered. This is fast, efficient, and allows future development to use a simple tweak to delay the animated image download, if desired.

  Notes: *Tumblr always provides "poster" elements on GIFs, so like the current code does this will erroneously add hoverable GIF labels to non-animated GIF images.

• When this code finds a possibly-animated image without a "poster," including webp images, it will download the image source, detect whether it's actually animated (if supported), create a blob URL containing a paused version of the image, and apply CSS that sets the image's content property to the new URL unless it's hovered. This enables only-if-animated webp image pausing, and using CSS instead of a canvas element means that future development can target more elements where the canvas would break layout (e.g. animated blog headers).

  Notes: Animated image detection requires the WebCodecs API, and thus requires Chromium or Firefox 133+. This is feature detected, and fallback createImageBitmap-based code that assumes .gif/.webp images are animated but otherwise fully works is included.

• When this code finds an element with a possibly-animated image url as part of its background-image property, it will create a paused image url as just described, and apply CSS that replaces the url in the image's background-image property unless it's hovered. This pauses elements we previously background-overrode like the tag page banner (including its gradient!), and means that future development can target even more elements that use this property.

  Notes: This uses a fairly complex regular expression. Elements are processed faster in Chromium or Firefox 133+, often pausing long before they finish downloading, because source URLs are available immediately and the WebCodecs API supports streaming.

This currently never calls URL.revokeObjectURL. Some rudimentary investigation appears to reveal that blob URLs are stored on-disk (so I don't think you can out-of-memory your browser with this) until the page is hard-navigated.

Here's (iirc) every technique I considered (diagram courtesy of https://mermaid.js.org/intro/):

details

We can insert paused content as:

• A canvas element (currently implemented). Doesn't work on elements with background-image and certain image elements like headers (dom layout issues; possibly solvable).
• CSS content/background-image replacement with a blob URL. Highly compatible; a bit slow/expensive (blob URL creation is async and uses storage/memory).
• The poster element that's already there. Only works when Tumblr provides one, but enables a cool delay-loading-until-hovered feature.

We can create paused content via:

• Copying the target image. Doesn't work on elements with background-image and doesn't help distinguish animated/non-animated webp.
• Downloading the target image's source. Expensive, obviously (but pretty fast if you're a bit clever).
• Using the poster element that's already there; only works when Tumblr provides one, as mentioned.

The vast majority of elements we care about are GIFs with posters, so we want an efficient method for those.

The download and css code paths are necessary to cover everything, but we definitely want to avoid downloading in high volume, so we want at least one more path. I picked use poster because I like the delay-load feature and it's efficient and simple, and declined to add anything else to reduce the LOC count; there are other combinations with minor upsides, but imho they're mostly worse.

diagram source:

flowchart LR
  gif-with-poster-->useposter;
  gif-with-poster-->copyposter-canvas;
  gif-with-poster-->copyposter-css;
  gif-with-poster-->copy-canvas;
  gif-with-poster-->copy-css;
  gif-with-poster-->download-canvas;
  gif-with-poster-->download-css;

  gif-->copy-canvas;
  gif-->copy-css;
  gif-->download-canvas;
  gif-->download-css;

  webp-->download-canvas;
  webp-->download-css;

  gif-nodom-->copy-css;
  gif-nodom-->download-css;
  webp-nodom-->download-css;

  backgroundgif-->download-css;
  backgroundwebp-->download-css;

misc notes

small load speed improvement for non-poster animated images:

  const getCurrentSrc = gifElement =>
    gifElement.currentSrc ||
+   gifElement.srcset?.split(',').at(-1)?.split(' ').filter(Boolean).at(0) ||
    new Promise(resolve => gifElement.addEventListener('load', () => resolve(gifElement.currentSrc), { once: true }));

  const pauseGif = async function (gifElement) {
-   await loaded(gifElement);
-   const pausedUrl = await createPausedUrl(gifElement.currentSrc);
+   const pausedUrl = await createPausedUrl(await getCurrentSrc(gifElement));
    if (!pausedUrl) return;

~~A dev dependency is added just to get intellisense on the WebCodecs APIs; this is unnecessary if the developer is using Typescript 5.8, which can be enabled in VS Code by installing the "JavaScript and TypeScript Nightly" extension, and will of course eventually be unnecessary in general.~~ Edit: VS Code's latest release updates typescript, making this unnecessary.

Testing steps

[marcustyphoon]

Ah, here's an interesting oddity: The WebCodecs API ImageDecoder in Firefox 135 appears to need to wait for the full image data to be available to decode a webp image, but will decode the first frame of a gif image as soon as it has enough data. In Chromium, both are decoded ASAP.

We actually have—to an extent—the ability to choose whether we fetch a webp or gif in many cases, because gifv is served as either depending on the fetch headers. I committed fetch(sourceUrl, { headers: { Accept: 'image/webp,*/*' } });, which will generally result in gifv files being served as webp and matching the fetch performed by the browser to actually render the source. In Firefox, though, this makes webp gif (and Tumblr TV page gif) pausing noticeably slower.

What I don't know is how exactly caching works in this case, and thus whether there's a benefit to ensuring that we fetch the same data twice or whether we ought to either prioritize the gif via q-weighting or just omit the header.

In general, like in the "speed up XKit Rewritten's initialization on page load" project, it's possible to get quite into the weeds eking out every last frame of performance, but particularly because this PR already makes pausing instant for the vast majority of GIFs the user will see, I certainly don't think it's worth additional effort. But I did test it, so I'll note it.

(somewhat relevant: https://developer.mozilla.org/en-US/docs/Web/HTTP/Content_negotiation/List_of_default_Accept_values#values_for_an_image)

[marcustyphoon]

Ooh, another one, but this one's important: the double-processing logic here is broken in the scenario where the image is removed from the DOM and re-added later to the same containing element. I made it decline to reprocess if the label's there, but that made sense only when the result of the processing is a canvas element that's still there.

• [x] fix this

Discovered in an extremely specific niche scenario: scrolling a recommended tags carousel element (https://www.tumblr.com/explore/recommended-for-you) with a gif in it off of and back on to the screen... only in the phone viewport width; different code is used in regular widths that doesn't detach the element. I mean, come on, man.

[marcustyphoon]

Now this one is fascinating.

In Safari, this fails to show the image behind the poster when the user hovers and invalidates the rule:

const hovered = `:is(:hover > *, [${hoverContainerAttribute}]:hover *)`;

img:has(~ [${posterAttribute}]):not(${hovered}) {
  visibility: hidden !important;
}

Moving the attribute to the animated image instead of the poster* and changing out that :has() for a regular ~ made it work just fine... and so does just doing this:

- img:has(~ [${posterAttribute}]):not(${hovered}) {
+ img:has(~ [${posterAttribute}]:not(${hovered})) {
    visibility: hidden !important;
  }

This is thus almost certainly a bug in Safari's CSS invalidation logic around the :has() and :is() selector combination (I tested by showing both and just changing a border color, so it's not hover/visibility/rendering related). The original version works with :hover and :is(:hover) but breaks with a complex selector... could it be that multiple complex selectors on the same "level" break?

Maybe at some point I'll make a minimal demo of this and submit it to... something, I dunno.

*ideally we don't do this, as on mobile devices Tumblr's code sometimes deletes the animated image and keeps the poster, as noted in the previous now-outdated comment.

[marcustyphoon]

High-effort anti-memory-leak code. ~~There are easier ways to prevent this~~ edit: hm maybe not actually, but having a use for WeakRefs is fun.

const cache = {};

const createPausedUrl = (sourceUrl, element) => {
  cache[sourceUrl] ??= { refs: [], result: createPausedUrlInternal(sourceUrl) };
  cache[sourceUrl].refs.push(new WeakRef(element));
  return cache[sourceUrl].result;
};

const clearCache = () => {
  let cleared = 0;
  for (const [sourceUrl, { refs, result }] of Object.entries(cache)) {
    if (!refs.some(ref => ref.deref())) {
      cleared++;
      delete cache[sourceUrl];
      result.then(url => URL.revokeObjectURL(url));
    }
  }
  cleared && console.log(`cleared ${cleared} cached paused GIF blobs (kept ${Object.keys(cache).length} currently active)`);
};

const ONE_DAY = 24 * 60 * 60 * 1000;
setInterval(clearCache, ONE_DAY);

[marcustyphoon]

This is good reference for me, but it can be good reference closed too.