A/V desync while using wireless sound device

[noctulescent]

mpv version and platform

macOS 10.12.3 0.24.0

Reproduction steps

Set sound output to a wireless sound system, open a video file.

Expected behavior

The video is synched to the delayed audio output from the wireless device.

Actual behavior

Video starts immediately regardless of the delayed audio, resulting in a 2~3 seconds desync.

setup : Speakers are plugged in the home gateway. Laptop is connected to said gateway via wi-fi. Set laptop's sound output to the gateway's server, system preferences says its type is AirPlay. Sound works but suffers a delay of 2~3 seconds.

additional infos : Viewing any youtube video using Safari shows the expected behavior. However, using Chrome doesn't. It's not just Chrome tho ; from the top of my head, skype and any game launched from steam in recent memory had the same issue. I suppose it's a feature from Safari or has to do AirPlay (but I probably have no clue what I'm talking about).

[rgaufman]

Any solution to this? - I have the same issue trying to airplay to a HomePod

Edit: seems setting the delay exactly -2000ms works great with HomePod!

[ehrlichja]

I'm having this exact issue too. Wireless is not required though - its happening to me when i Airplay to my HifiBerry using Ethernet. The sync lag is exactly -2000ms. MacOs 11.2 Big Sur, mpv 0.33.0.

[Akemi]

also see https://github.com/mpv-player/mpv/issues/9243#issue-1007339900 for more additional info.

[codyps]

Looking at the MPV code and Apple docs, here's my thoughts on where this issue might be originating:

Lay of the land:

• for video output, mpv is using the corevideo api. This API gives mpv a callback (CVDisplayLinkOutputCallback) which provides mpv with a timestamp for the frame it needs. mpv ignores this timestamp and instead provides the next frame. More detail: in mpv the callback displayLinkCallback calls reportRenderFlip which calls mpv_render_context_report_swap which updates flip_count and broadcasts on the video_wait cond var. The video_wait condvar is waited on by both flip_page and mpv_render_context_render, and variously signaled/broadcasted by the same. flip_page, however, is actually waiting on flip_count to change. The intent appears to be that flip_page triggers rendering the frame via incrementing present_count and broadcasting on video_wait, then waits for rendering to happen separately via mpv_render_context_render.

• separately, mpv is handling audio via coreaudio. It's obtaining latency via ca_get_device_latency_us, which is summing the device properties kAudioDevicePropertyLatency, kAudioDevicePropertyBufferFrameSize, kAudioDevicePropertySafetyOffset. Missing here is handling of kAudioStreamPropertyLatency (which it appears that chrome and others are incorporating into their latency values)

Possibilities (possibly a combination of these):

1. CVDisplayLinkOutputCallback is giving us timestamps in the "past" and expecting frames rendered for them, but is instead getting too-new frames from mpv.
2. not using kAudioStreamPropertyLatency is causing us to underestimate audio latency.
3. we only call ca_get_device_latency_us when initializing audio, if the latency changes after the ao_driver has been initialized (say, due to a bluetooth device connecting or the user switching the output in macos to an airplay device), we'll still be using the old audio latency (the reproduction steps for this issue indicate this last one might not be an issue for this case, but is likely an issue in other cases).

[rgaufman]

The above is great investigation and seems like a great plan!

[krackers]

Seems like this is particular to airplay? I see the 2000ms mentioned in https://bugzilla.mozilla.org/show_bug.cgi?id=776137 as well

And looking at the chromium bug https://bugs.chromium.org/p/chromium/issues/detail?id=783282 the cause is due to ignoring the stream latency component

[forthrin]

+1

PS! I thought the reason for this was that there is latency in Bluetooth audio, and thus when jumping to a new position in the video, playback has to wait the same amount of milliseconds for video and audio to be in sync.

I thus assumed a possible workaround would be to start video playback immediately, but play audio with a inverse delay. (A minor audio dropout would be better than video halting, if one had to choose.)

Maybe the problem is unrelated to this. At any rate I hope it will be fixed, as it's quite annoying when jumping around.

[Akemi]

this is possibly fixed by https://github.com/mpv-player/mpv/commit/2fa1e7d0b409f2ceb608d9b6c5d90106507595aa

can anyone confirm?

[forthrin]

Tried it just now. But might mine be a different issue? My observation with AirPods Pro 2 (Bluetooth):

1. Press → or ↑ to jump ahead in video.
2. Video jumps ahead instantly, but does not play immediately.
3. After 1 second or so, the video plays, and the audio is in sync.

Do note:

1. The error does not happen consistently. I see no pattern to when it happens.
2. If disabling video, the delay still happens with the audio.
3. If disabling audio, the delay is near-zero.

Note also:

1. With QuickTime Player (with Bluetooth), the delay is near-zero.
2. With wired speakers (without Bluetooth), the delay is near-zero.

The 1 second delay becomes very annoying when pressing the arrow keys multiple times to skip ahead to something.

Let me know if this is a different issue, if so, refer to an existing bug report, if any. If not, I can create one.

[Akemi]

i assumed that this and #9243 are the same problem. though this issue only reported an initial general delay. if that issue is fixed and the issue reported in #9243 still persists, we should close this one and reopen #9243.

[rgaufman]

This is only specific to wireless, e.g. a HomePod. This does not apply to Bluetooth at all.

[Akemi]

bluetooth is a wireless technology too. either you mean wifi/wireless lan or ethernet in general.

[rgaufman]

Wireless as in WiFi, specifically airplay to HomePod (which does not have an Ethernet version so not sure if Ethernet is impacted).

[Dudemanguy]

This was possibly fixed by 39f7f83351524cdbd87c5ae420b838530314b525. Can anyone confirm?

[PhilipMok-cc]

Does this issue could be solved by modifying the core audio out ?

[Dudemanguy]

Anyone know if this still happens? The previously linked commit is in 0.36 now.

[forthrin]

I have no recollection of having been bothered with this the last half year or so.

[Dudemanguy]

In that case, I will close this.

[aleksfadini]

On my machine (M1 mini), mpv will not sync correctly to Apple HomePods