Video Block: First frame not shown as the poster on iOS Safari — Fix: Use Fragment URL

[porg]

Bug and workaround widely known

• 2019-06: https://github.com/video-react/video-react/issues/328
• 2019-06: No way to preload first video frame on iOS Safari?
• 2011: In the early HTML5 days other browsers also did yet not supported this
  • Update May 2021-05 — Most browsers fixed it, load the first frame and don't need that enforcement hack anymore.

Officially reported in WebKit issue tracker too

• Mobile Safari has a bug reported since 2019-05-06

It is currently not possible to render the first frame of the video as the poster image on iOS Safari when using the native <video> element. Interestingly:

• macOS Safari renders the first frame without issue, probably because it automatically uses "auto" for preloading
• iOS Safari can render the first frame without issue if the video is autoplaying

However, if the video is not set to autoplay on iOS Safari, the first frame of the video is never fetched. Setting preload="auto" on the element does not work, as iOS Safari currently only supports the more restrictive "metadata" value when it comes to preloading: https://webkit.org/blog/6784/new-video-policies-for-ios/

Solution aka Quick Fix: Append Fragment URL #t=0.001

Hack for iOS Safari to display the HTML video thumbnail (by MuffinMan.io, in 2020-05) in essence:

<video><source src="path-to-video.mp4#t=0.001" type="video/mp4" /></video>

Proposed Fix for Wordpress Video Block

• The video block appends the fragment URL #t=0.001 to the video's src for the combination of playback options where this makes sense (you experts know that certainly)
• Fragment URLs seem to be widely supported, so this fix targeting Mobile Safari likely has no undesired effects elsewhere.
• Or at least an option under WordPress → Settings → Media
  • Add section "Video" with option
  • ☑︎ Ensure that Mobile Safari loads 1st frame when using preload. (More info)

Reproduction

1. In the Block Editor insert a video block into a page or post.
   • Video source: MP4 with a H264 video codec, no audio track.
   • Settings for the Video block
     • Autoplay: OFF
     • Loop: OFF
     • Muted: ON
     • Playback controls: ON
     • Play inline: ON
     • Preload: Tried all 3 settings: Auto | Metadata | None — Made no difference for the bug!
     • Poster Image: None set — I want myself to spare this, and let the web browser load the first frame as its poster (First HTTP request with a byte range 0-1 for the moov atom (aka quickstart) and then a second request with a byte range of few KB for the first frame)
2. Save the page.
3. Open the page on an iPhone (I used Mobile Safari 15.6.3 on iOS 15.7.5)
   • Actual: You only see the Play button (white triangle on grey circle)
   • Expected: You also see the first frame of the video.

Workaround — Manually add media fragment to URL of every video block instance on your website

1. Add video to Media Library (optional)
2. Insert video block and choose video from the Media Library or externally hosted video URL.
3. Select video block → Toolbar → Replace → Current media URL → Edit
4. Append your Media Fragment #t=0.001 to the URL and then APPLY.
5. Save page
6. Reload in Mobile Safari: ✅ First frame now shown as the poster.

WordPress information

• WordPress version: 6.2.2
• Gutenberg version: 15.9.1
• Are all plugins except Gutenberg deactivated? No
• Are you using a default theme (e.g. Twenty Twenty-One)? Yes

Affected devices / versions

• iOS 15-16
• Affects elder devices such as my own: iPhone 6S, iOS 15.7.5, Mobile Safari 15.6.3
• Affects also newest iOS and devices: iPhone 14 Plus, iOS 16, Mobile Safari (tested on Browserstack.com)

[fluiddot]

Hey @porg 👋 , thanks for reporting the issue. The "Mobile App - i.e. Android or iOS" label is usually set for issues affecting the WordPress/Jetpack app. Based on the issue's description, I understand that it's related to the web editor when using the mobile browser, is this accurate? If so, I'd consider removing the label and maybe using a different one like "Mobile Web". Thanks!

[porg]

I'm just a reporter, not staff. Could you please label/move the issue as it should be? Thanks!

[fluiddot]

I'm just a reporter, not staff. Could you please label/move the issue as it should be? Thanks!

Sure, I'll update the labels. Thanks!

[porg]

• Luckily found a fix which everyone can apply until this is fixed for the block.
• In the issue description I added a corresponding section with text and screenshot :
  • Workaround — Manually add media fragment to URL of every video block instance on your website
• But of course this should be fixed on the block level, so that one can simply insert a video block and be done with it, no need to manually fix this per each video block instance.

[porg]

Update: Affects also newest iOS and devices: iPhone 14 Plus, iOS 16, Mobile Safari (tested on Browserstack.com)

Updated "Environment" info in issue description.

[Infinite-Null]

Hi @porg 👋, I have opened a PR with the solution Can you please test it at your convenience. Thank You!

[porg]

Thx! I try to test it on the weekend. And then report back to you.

[porg]

@Infinite-Null Question: Is your solution which utilizes video.load() as mobile data efficient as possible?

• First HTTP request with a byte range 0-1 for the moov atom (aka quickstart)
• and then a second request with a byte range of few KB for the first frame
• Not more than this.

[porg]

@Infinite-Null I have no local test environment.

• I have not followed Gutenberg/WordPress developments the last 1-2years.
• Is it a good practise to test a PR on https://playground.wordpress.net/gutenberg.html
• I think I would test it there. Okay?

[Infinite-Null]

@Infinite-Null Question: Is your solution which utilizes video.load() as mobile data efficient as possible?

• First HTTP request with a byte range 0-1 for the moov atom (aka quickstart)
• and then a second request with a byte range of few KB for the first frame
• Not more than this.

Hi @porg, Yes, it is. The solution using video.load() is mobile data efficient. But, I think We can further optimize it by checking for iOS/iPhone using user agent detection and then applying load() conditionally, something like this:

const isIOS = /iPad|iPhone|iPod/.test(navigator.userAgent);
if (isIOS) {
    video.load();
}

This ensures the fix only runs on devices where the blank first frame issue actually occurs. This was not working on simulator as they have same user agent as Mac.

Or we can target AppleWebKit in the user agent? What do you say 🤔?

I think I would test it there. Okay?

Ya sure 😄.

[porg]

1. Good that you confirm that video.load() is indeed mobile data efficient.
2. Narrowing it down to the user agent (Mobile Safari) which needs this fix is an ideal choice. Do not bother the other browsers which do it right already. Only serve that fix to those which really need it.
3. I will test your revised version on https://playground.wordpress.net/gutenberg.html with my real iPhone and give you the most accurate feedback then. Ping me when ready for testing.

[porg]

Btw I recently saw the elegant minimal RegEx iP\w+ which captures all Mobile Apple devices 😉 but yes your taxative browser match is preferred because then the intent gets very clear.

[Infinite-Null]

Hey @porg, While working on this, I found something interesting the user agent of Chrome was:

Reference: Chrome User Agent Guide

It turns out that relying on AppleWebKit isn’t quite reliable in this case. Instead, I used your regex iP\w+, and it worked really well! 😄

https://github.com/user-attachments/assets/aab970db-d613-4de4-8fa8-861abe3eccdb

Would really appriciate if you could also test it and provide the feedback 🙌.

Thank you!

[porg]

Sorry, did not get to it yet, but very likely will now this weekend!