[playback] Seamless branching files drop frames on merging spot

[mitzsch]

Important Information

Information:

• mpv version: 0.35.0-348-g0f13c38e
• Windows Version: Win11 x64
• Source of the mpv binary: shinchiro mpv-winbuild

Reproduction steps

Use any file that was stitched together (two or more small file parts) in the process of creation. In my case, this is a 4k blu ray where the movie is split into many small parts due to having both an extended and theatrical version on one disc. Playing back such a file leads to unexpected behavior. In my case whenever such a "merging spot" (the spot where the first part ends and the second part starts) is hit the frame drop counter gets reset. This happens regardless of displaying framerate. The other issue comes when you play the file at its own (resolution)framerate. In this case 2160@23pHDR. In this scenario, many/visible frame drops occur when the merging spot is being hit in addition to the frame drop counter reset.

I created a small sample, that shows the issue. mpv.com file-reset-framedrop.mkv -vo=gpu-next -hwdec=d3d11va -v -msg-level=all=v,ffmpeg/video=status

In the log you can see this one

[cplayer] VO: Description: Video output based on libplacebo
[vo/gpu-next] reconfig to 3840x2160 d3d11[p010] bt.2020-ncl/bt.2020/pq/limited/display SP=4.359606 CL=uhd
[vo/gpu-next/win32] reset window bounds: 961:31:958:1000
[vo/gpu-next] Window size: 958x1000 (Borders: l=0 t=0 r=0 b=0)
[vo/gpu-next] Video source: 3840x2160 (1:1)
[vo/gpu-next] Video display: (0, 0) 3840x2160 -> (0, 231) 958x538
[vo/gpu-next] Video scale: 0.249479/0.249074
[vo/gpu-next] OSD borders: l=0 t=231 r=0 b=231
[vo/gpu-next] Video borders: l=0 t=231 r=0 b=231
[statusline] AV: 00:00:03 / 00:00:06 (51%) A-V:  0.004

It looks like a completely new file has been opened...

Expected behavior

Just play the file without introducing additional frame drops and do not reset counters.

Actual behavior

When the merging spot is reached and played, frame drops occur and the framedrop counter is reset.

Log file

file-reset.txt (I just cleaned it a bit, I only deleted some [ffmpeg/video] hevc: Skipping NAL unit 63 messages...)

Sample files

I hope file.io is okay... The link will expire in about 4 months. Inside the zip, you will find the mkv file file-reset-framedrop.zip

[mitzsch]

Update on that one

I now had some "rips" that showed this behavior - all of them stem from discs that build up the feature film from many smaller parts.

I re-uploaded the sample => https://fastupload.io/ApvdE8ewMo1ndpo/file - valid for 4 months

Here are the logs I created. Notice this happens with vo=gpu and vo=gpu-next vo=gpu: mpv-drop-spot-gpu.conf.txt vo=gpu-next: mpv-drop-spot.conf.txt

I also noticed these lines here =>

[   3.497][v][vd] Decoder format: 3840x2160 d3d11[p010] bt.2020-ncl/bt.2020/pq/limited/auto SP=4.359606 CL=uhd crop=3840x2160+0+0
[   3.498][v][vd] Using container aspect ratio.
[   3.498][v][vf] [in] 3840x2160 d3d11[p010] bt.2020-ncl/bt.2020/pq/limited/display SP=4.359606 CL=uhd crop=3840x2160+0+0
[   3.498][v][vf] [userdeint] 3840x2160 d3d11[p010] bt.2020-ncl/bt.2020/pq/limited/display SP=4.359606 CL=uhd crop=3840x2160+0+0
[   3.498][v][vf] [autorotate] 3840x2160 d3d11[p010] bt.2020-ncl/bt.2020/pq/limited/display SP=4.359606 CL=uhd crop=3840x2160+0+0
[   3.498][v][vf] [convert] 3840x2160 d3d11[p010] bt.2020-ncl/bt.2020/pq/limited/display SP=4.359606 CL=uhd crop=3840x2160+0+0
[   3.538][v][autoconvert] Sub-filter requires draining but we must destroy it now.
[   3.538][v][convert] dropping frame due to pin disconnect

Especially the last one... Unfortunately, I have no idea how to interpret these. What is it converting? Can this be optimized? May I ping you @haasn?

[mitzsch]

Update on that one. I tried playback of these files in different players. ffplay and VLC just play them just fine... This seems very strange that such a nice player like mpv has a problem with playing them back without hick-ups.

@Dudemanguy
Sorry to ping you all... But I have seen you have done some work on filters. May I ask you to have a look at this one?

[mitzsch]

Update: Tried the lavf demuxer instead of the default mkv demuxer as I thought it would make a difference but nope... no difference

[mitzsch]

Hm, bump, seems nobody has an interest in this one or no idea how to fix it... Sad... Any other player plays those files fine... I will need to test Kodi...

The issue is the only real showstopper for me to call this player "perfect"...

I will also try the workaround of reencoding those few movies, then the issue should vanish. Luckily handbrake supports HDR reencodes without quality loss. (Downside, reencoding takes ages and I mean ages...)

[mitzsch]

I probably found the culprit. It's the changing HDR metadata that is attached to the files. To be precise it affects maxCLL and maxFALL:

Here are some examples:

First part: Second part:

Different file:

First part: Second part:

Different attached metadata that causes severe frame drops. This also explains why SDR files are unaffected (no lum metadata) and why other UHD rips with seamless branching work fine. For example, Disney rips are almost always seamless branching discs but they usually don´t have any maxCLL or maxFALL metadata attached and the other lum values stay the same. No dropped frames as a result.

Sorry to ping you again @haasn but I guess you are the person to ping in this case. There is probably no option to ignore any metadata that comes after the first few frames of file or that only uses the metadata attached to the header?