Hanging in some videos using opus codec

[christoph-heinrich]

Important Information

Provide following Information:

• mpv version: git master (602995fd40f03440f5fa64113b6188cb46ffcc9c) compiled and installed with the mpv-git AUR package
• Linux Distribution and Version: Arch
• Source of the mpv binary: compiled from source
• If known which version of mpv introduced the problem: unknown
• Window Manager and version: kwin 5.24.5
• GPU driver and version: Mesa 22.1.1 AMD R9 380

Reproduction steps

mpv --no-config --af-add="@scaletempo:scaletempo=stride=28:overlap=.9:search=25" --speed=2 --ytdl-format=251 --start=+120 --length=4 "https://www.youtube.com/watch?v=rDU1i3RPgZg" results in hanging with a single CPU core going to 100% and the ouput [cplayer] Audio device underrun detected.. This does not happen with all videos and I have only observed it when using the opus codec on YouTube videos. The --af-add is not strictly nesessary, however this configuration makes it much worse then the default scaletempo2 (I can't notice the problem anymore on scaletempo2 when I allow my CPU to clock to it's max frequency, but there is still a short spike in cpu usage). In the log file pulseaudio was used, but I also tried --ao=alsa and --ao=pipewire with the same result. It does not matter if video is played in addition to the audio. mpv --no-config --af-add="@scaletempo:scaletempo=stride=28:overlap=.9:search=25" --speed=2 --ytdl-format=251 --start=14:39 --length=5 "https://www.youtube.com/watch?v=YdfeRAAnQLU" is another particularly bad example for me.

Expected behavior

Smooth playback as usual

Actual behavior

Hangs for short period

Log file

output.txt

Sample files

The videos from SemperVideo all cause this problem to some degree, but I found a particularily bad section in this one for the reproduction. It is worth noting that this is not exclusive to SemperVideo, they are just very reliable in causing this. https://www.youtube.com/watch?v=YdfeRAAnQLU for example also causes this at at 14:40.

[Dudemanguy]

For whatever it's worth, the example command you linked doesn't hang for me.

[christoph-heinrich]

I just tried it on my laptop with an Intel i5-6300U and it only hangs a little bit (barely noticable, but I still get [cplayer] Audio device underrun detected.) when locked to the lowest frequency, where as on my desktop with an AMD FX-8350, my example hangs even when locked to max frequency.

Any ideas on how i could figure out where the problem lies?

I've been using the git version of mpv for a long time now and I'm pretty sure there has been a short period where this wasn't a problem, but I don't remember when this was, and I already tried all commits back to 04.10.2021 to find it. I can't go back further then this because there was some change with libplacebo and it doesn't compile anymore. I remember looking at the latest commits out of curiosity, back when this was resolved, and one of them was something about inserting zeros when there is no sound, or something like that. But I tried looking for it with git log --grep= without success, so maybe my memory is wrong.

That "inserting zeros" thing sounds kind of like audio-stream-silence, but that didn't help.

[christoph-heinrich]

I forgot to mention, this only seems to happen when there is some silence. Usually because the person stopped talking for a little bit and there is no background music. This made me wonder if it maybe only happens when the video producer uses a noise gate, so that periods of silence are actually silent, where for most videos there would still be a small amount of background noise, quiet enough to not be noticeable.

[Dudemanguy]

I don't think there were any commits anytime in recent months that would be related to this.

[christoph-heinrich]

It could be well over a year ago at this point. I just didn't have time for that back when it happened and now I can only go back so far. ~Also for all I know, maybe there just happened to be some changes in ffmpeg or some other dependency that influenced that. Would be interesting to find out though.~

[richardpl]

Why you think that? FFmpeg have nothing to do with your issues. Perhaps filter just cause more denormals to pop up than usual.

[christoph-heinrich]

~Afaik the filters are from FFmpeg and since scaletempo=stride=28:overlap=.9:search=25 is more prone to cause hanging for me then scaletempo2, I figured it might have something to do with FFmpeg. But I don't really know how things actually work internally, just speculation on my part.~

[christoph-heinrich]

Just found another example that causes severe hanging for me and added it to the description.

[richardpl]

If you bothered to check facts, you would notice that filters you talk about are not originating from FFmpeg at all. But just continue spreading untrue and harmful statements as usual on internet its all normal.

[christoph-heinrich]

I didn't look at the code and the man page says most filters use libavfilter, thus the assumption that FFmpeg might have something to do with this.

But now I did look at the code and through the power of printf debugging and measuring time with sys/time.h found out that this loop https://github.com/mpv-player/mpv/blob/c961f4d0dbea51d6a0c74725eadd65d21a4701bf/audio/filter/af_scaletempo.c#L157-L158 usually takes 8-9μs. However when this hanging occurs the loop takes between 230μs and 820μs. Additionally the loop time always hovers around the same time between each Audio device underrun detected., resulting in an output that looks like this:

...
333
361
332
333
345
332
return 0

Audio device underrun detected.
69500:02:01 / 00:04:07 (49%) x2.00
721
700
695
692
700
699
...

No idea why that happens and how I could change that loop to prevent it.

Edit: Found a difference between opus and m4a ! Accumulated all samples and output the sum at the end of the function. For m4a I get 0, but for opus I get a non 0 number (e.g. 4.06264450777050965202e-39) during silence (when it hangs).

[christoph-heinrich]

As requested in IRC, here is the output of mpv --no-config yourclip -o test.opus (keep in mind the hang refered to in Reproduction steps happens at 120 seconds)

[christoph-heinrich]

Also happens with local files

[haasn]

From my impression of the IRC debugging, and the mention of extremely low sample values in the loop (as low as 4.06264450777050965202e-39), I think this article is relevant: https://en.wikipedia.org/wiki/Subnormal_number#Performance_issues

I do think we should probably round denormalized audio samples to zero. The question remains, to me, where the hell are these values coming from and which layer is responsible for rounding them? Do they come from the actual file somehow? Are they introduced by af_scaletempo? Or possibly some other component?

And also, what would be the best way to round them away? Explicit if (issubnormal()), or using platform-specific and/or custom asm code to enable automatic FPU rounding to zero?