mpv + yt-dlp + sponsorblock + seek = no new video frames

[Obegg]

Important Information

Provide following Information:

• mpv version:

mpv v0.37.0-56-g67aa5684 Copyright © 2000-2023 mpv/MPlayer/mplayer2 projects
 built on Dec  2 2023 12:38:53
libplacebo version: v6.338.0-60-g795600a-dirty
FFmpeg version: N-112889-g5230257ea
FFmpeg library versions:
   libavutil       58.32.100
   libavcodec      60.35.100
   libavformat     60.18.100
   libswscale      7.6.100
   libavfilter     9.14.100
   libswresample   4.13.100

• Windows Version:

Edition	Windows 10 Enterprise LTSC
Version	21H2
Installed on	‎21/‎10/‎2023
OS build	19044.3693
Experience	Windows Feature Experience Pack 1000.19053.1000.0

• Source of the mpv binary: https://github.com/zhongfly/mpv-winbuild/releases
• If known which version of mpv introduced the problem: No
• Possible screenshot or video of visual glitches: No

Reproduction steps

While using mpv (zhongfly) + yt-dlp (master) + mpv_sponsorblock (PR 71 which skips poi_highlight) it seems like when seeking on this YouTube video as example (doesn't matter if it's auto seek or a manual seek) then yt-dlp will not show me any new frames from the video and I can only hear the audio track, or in some cases the stream itself will not load.

For testing purposes I've used mpv portable_config directory with the following commands on mpv.conf:

no-config
log-file="~~/yt-dlp-mpv-seek-issue.txt"

And added sponsorblock script with the following options on sponsorblock.conf:

categories=sponsor,selfpromo,interaction,intro,outro,preview,music_offtopic,filler,poi_highlight
skip_categories=sponsor,selfpromo,interaction,intro,outro,preview,music_offtopic,filler,poi_highlight

Note 1: Some information on the log file I've changed to GitHub. Note 2: This issue has been bothering me for quite some time so it's not a new bug. Note 3: The bug can be reproduced even without sponsorblock script, just a manual seek at the start of the stream to the middle of the stream. Note 4: Created the same issue over at yt-dlp issue tracker.

Expected behavior

Show new video frames.

Actual behavior

No new video frames, only audio.

Log file

yt-dlp-mpv-seek-issue.txt

Sample files

portable_config.zip https://www.youtube.com/watch?v=2QI88aLyaOs

[Obegg]

Seems like this issue doesn't get any attention, so allow me to clarify.

When playing a video file (not a local file) using yt-dlp and then seeking forward (could be auto-skip using scripts and could be a manual skip) the result will be one out of two:

1. The video will seek to the desired point, but no new video frames will draw, only audio will be heard.
2. The video will seek to the desired point, but will not load either video and audio.

This doesn't seem to be an issue with yt-dlp, because I did open the same issue there and they believe it's mpv issue. The logs show "EOF reached" while encountering the first outcome I mentioned:

[  11.665][v][lavf] EOF reached.
[  11.665][v][timeline] switch to segment 0
[  11.665][v][mkv] queuing seek to 540.790000
[  11.665][v][mkv] execute seek (to 540.790000 flags 32)
[  11.665][v][ffmpeg] stream level seek from 98304 to 9184314
[  11.665][d][ffmpeg] tcp: Starting connection attempt to IPv6 port 443
[  11.669][d][ffmpeg] tcp: Successfully connected to IPv6 port 443
[  11.692][v][mkv] seek done
[  11.692][v][timeline] seek done
[  11.692][d][timeline] stream 0: resize index to 128
[  11.740][d][timeline] stream 0: resize index to 256
[  11.770][d][timeline] stream 0: resize index to 512
[  11.809][v][mkv] EOF reached.
[  11.809][v][timeline] EOF reached.
[  11.809][v][vf] filter input EOF
[  11.809][v][vf] filter output EOF
[  11.809][v][cplayer] video EOF reached
[  11.809][d][cplayer] video EOF (status=4)
[  11.812][d][cplayer] video EOF (status=4)
[  11.812][v][cplayer] audio ready
[  11.812][d][cplayer] video EOF (status=4)
[  11.812][v][cplayer] starting audio playback
[  11.812][d][ao/wasapi] Thread Resume
[  11.812][d][ao/wasapi] Thread Reset
[  11.812][v][cplayer] playback restart complete @ 540.790000, audio=playing, video=eof
[  11.812][d][cplayer] video EOF (status=4)
[  11.813][d][cplayer] video EOF (status=4)
[  11.813][d][cplayer] video EOF (status=4)
[  11.823][d][cplayer] video EOF (status=4)
[  11.823][d][cplayer] video EOF (status=4)
[  11.838][d][cplayer] video EOF (status=4)
[  11.838][d][cplayer] video EOF (status=4)
[  11.843][d][cplayer] video EOF (status=4)
[  11.843][d][cplayer] video EOF (status=4)
[  11.858][d][cplayer] video EOF (status=4)
[  11.858][d][cplayer] video EOF (status=4)
[  11.858][d][cplayer] video EOF (status=4)
[  11.858][d][cplayer] video EOF (status=4)
[  11.858][d][cplayer] video EOF (status=4)
[  11.858][d][cplayer] video EOF (status=4)

[mooreye]

I just rechecked and it seems that even without that sponsorblock LUA script, if you just click on a part of video on OSD that is not yet cached, only audio will be heard but no new frames will be displayed. So this seems to be an issue with mpv specifically.

I use yt-dlp version 2023.11.16 and mpv version 0.36.0.

Even reloading the file with the following script doesn't fix it:

function reload_file()
    local path = mp.get_property("path")
    if path ~= nil then
        local time = mp.get_property_number("time-pos")
        mp.commandv("loadfile", path, "replace", "start=" .. time)
    end
end

mp.add_key_binding("ctrl+r", "reload", reload_file)

[TheHardew]

For me the problem was ytdl-format set to prioritize the vp09 codec.

617     mp4   1920x1080   60    │ ~754.85MiB 5259k m3u8  │ vp09.00.41.08 5259k video only
303     webm  1920x1080   60    │  325.23MiB 2267k https │ vp9           2267k video only          1080p60, webm_dash

It might seem that format 617 is much better, but after downloading them, it turns out they are exactly the same.

If that doesn't help, you can debug by running mpv --no-config to test whether you can seek. If it works, you can then run just mpv and try to enable user scripts and config options inside of mpv.conf until it breaks.

[guidocella]

--cache=no fixes seeking youtube videos for some reason. If you want to keep the cache you can use one of methods described in https://github.com/mpv-player/mpv/issues/8655#issuecomment-1199733728

[geextahslex]

@guidocella cache=no doesn't fix it in my case

[guidocella]

Using av1 instead of vp9 in https://www.youtube.com/watch?v=PgHZ4itvgVs fixes seeking rather than being fixed by --cachie=no.

[geextahslex]

But most other videos run fine with VP9, there must be a some difference that triggers this issue

[guidocella]

I can reproduce being able to seek the AV1 stream and not the VP9 stream in ffplay so it's an ffmpeg issue.

[geextahslex]

As @Obegg is mentioning "frames" the only thing I can see is that the video that doesn't work has a weird FPS 29.97003. Videos with VP9 60/30/25 FPS work well

[Obegg]

To be honest I am quite happy I am not the only one experiencing this, this gives us more information about this issue since more users are experiencing it and thereby reaching to the solution quicker. I'm quite sure I already tested cache=no and it did not fix the issue (if it did I would have used it the second I found out it fixes the issue) I have not tested other formats, only the default, I would love to test new formats, if someone is willing to share the command of how to do so in mpv.conf that would be nice. the screenshot of the reply above me looks about right, the video will stop rendering new frames and I would only hear the audio (this is only one out of the two scenario which could happen as I mentioned 2nd reply) I have no idea if it's a ffmpeg issue or not, I also not sure how I can determine it by myself since I probably lack the knowledge of doing so.

[guidocella]

ytdl-format=(bestvideo[vcodec^=av01]/bestvideo)+bestaudio/best circumvents the issue.

[geextahslex]

To be honest I am quite happy I am not the only one experiencing this

@Obegg I felt the same as I wasn't sure if to make the issue in the first place.

ytdl-format=(bestvideo[vcodec^=av01]/bestvideo)+bestaudio/best circumvents the issue

@guidocella A workaround is always nice but VP9 and VP9 2 is youtubes workhorse. Going forward it would be nice to have this issue solved

Would it be possible to only switch to AV01 if the faulty framerate is detected?

[WPMGPRoSToTeMa]

@guidocella A workaround is always nice but VP9 and VP9 2 is youtubes workhorse. Going forward it would be nice to have this issue solved

Would it be possible to only switch to AV01 if the faulty framerate is detected?

As far as I can see it is broken only for vp09.AB.CD.EF (e.g. vp09.00.40.08) and this codec is not always preferred by yt-dlp, e.g. I also see vp9 is preferred in some cases. I use the following selector to filter out the buggy codec: ytdl-format=bv[vcodec!^=vp09.]+ba/b

[Obegg]

I have a more fundamental question here, which might sound off-topic: Isn't AV1 better than VP9? I remember reading that AV1 is the new codec which is better quality and lower size, so why it isn't the default for everyone? I remember seeing some commits on FFMPEG regarding AV1, haven't tested it out yet. But if it is the case and AV1 is the best option - why isn't it the default on yt-dlp or mpv for that matter? Am I missing something here?

[guidocella]

https://github.com/yt-dlp/yt-dlp/issues/5629

[geextahslex]

I'm not sure but for me I have a better 4k performance with VP9 than with AV01 AV01 VP9

[SnnUntz]

These can be used to solve the issue. ytdl-raw-options = format-sort="proto" or ytdl-raw-options = extractor-args="youtube:skip=hls" or ytdl-format = "(bv*+ba/b)[protocol^=http][protocol!*=dash] / (bv*+ba/b)"

[geextahslex]

ytdl-raw-options = format-sort="proto"

what does it mean/do? ^^'

[SnnUntz]

ytdl-raw-options = format-sort="proto"

what does it mean/do? ^^'

# Download best video available via direct link over HTTP/HTTPS protocol,
# or the best video available via any protocol if there is no such video
$ yt-dlp -f "(bv*+ba/b)[protocol^=http][protocol!*=dash] / (bv*+ba/b)"

# Download best video available via the best protocol
# (https/ftps > http/ftp > m3u8_native > m3u8 > http_dash_segments ...)
$ yt-dlp -S "proto"

[geextahslex]

@SnnUntz

These can be used to solve the issue. ytdl-raw-options = format-sort="proto" or ytdl-raw-options = extractor-args="youtube:skip=hls" or ytdl-format = "(bv*+ba/b)[protocol^=http][protocol!*=dash] / (bv*+ba/b)"

Thats amazing, all of them work. But what does it change? Why is it working like this? ^^'

[SnnUntz]

Thats amazing, all of them work. But what does it change? Why is it working like this? ^^'

The bug occurs when using the m3u8 protocol.

[WPMGPRoSToTeMa]

The bug occurs when using the m3u8 protocol.

It's important to note that only vp9 via m3u8 is bugged, e.g. h264 via m3u8 works fine.

[SnnUntz]

The bug occurs when using the m3u8 protocol.

It's important to note that only vp9 via m3u8 is bugged, e.g. h264 via m3u8 works fine.

Then this should work too: ytdl-format = bv*[vcodec!*=vp09]+ba/b

[geextahslex]

@SnnUntz

Then this should work too: ytdl-format = bv*[vcodec!*=vp09]+ba/b

Yes this works too, but you loose the ability to pick your resolution with something like ytdl-format=bestvideo[height<=?1440]+bestaudio/best

[SnnUntz]

After some research I found that this is an ffmpeg issue. https://trac.ffmpeg.org/ticket/7359 https://trac.ffmpeg.org/ticket/11006

[WPMGPRoSToTeMa]

Then this should work too: ytdl-format = bv*[vcodec!*=vp09]+ba/b

Yeah, I mentioned a similar fix previously https://github.com/mpv-player/mpv/issues/13033#issuecomment-2198678823.

Yes this works too, but you loose the ability to pick your resolution with something like ytdl-format=bestvideo[height<=?1440]+bestaudio/best

If you use uosc like me, then it can be fixed in their .lua code. But yeah, this is exactly the reason I currently use ytdl-raw-options = extractor-args="youtube:skip=hls".

[SnnUntz]

Yes this works too, but you loose the ability to pick your resolution with something like ytdl-format=bestvideo[height<=?1440]+bestaudio/best

If you use uosc like me, then it can be fixed in their .lua code. But yeah, this is exactly the reason I currently use ytdl-raw-options = extractor-args="youtube:skip=hls".

You can also use it this way: bestvideo*[vcodec!*=vp09][height<=?1440]+bestaudio/best

[veganomy]

This is more of a manifest.googlevideo.com issue than m3u8/hls/ffmpeg/yt-dlp. HLS playback seems fine on many websites to me.
It seems to happen only when the server doesn't request the specific HLS fragment when you seek on your favorite player. Some servers support third party players. Some don't. Instead, they force sequential fragment unlock only. YouTube seems to be the one which does that.

That's why @guidocella's piping method seems to work, because you're restricting yourself to load fragments in sequential order and avoiding seeking issues.

[ValeZAA]

After some research I found that this is an ffmpeg issue. https://trac.ffmpeg.org/ticket/7359 https://trac.ffmpeg.org/ticket/11006

That is fixed btw

[Dudemanguy]

Allegedly fixed upstream.