Jittery playback with gpu-next when starting 60fps video paused with 2x speed

[christoph-heinrich]

Important Information

Provide following Information:

• mpv 0.34.0-386-g284fecc0bd
• Arch Linux
• compiled form source (latest master)

Reproduction steps

1. Download haasns 60 fps test video
2. run mpv --no-config --speed=2 --pause --force-window=immediate --vo=gpu-next --video-sync=display-resample --interpolation ~/Downloads/60.000.mkv
3. reduce speed to 1 (e.g. by pressing backspace)
4. start playback (e.g. by pressing space)

For me the playback looks like this (recorded with obs)

https://user-images.githubusercontent.com/8932183/182045512-cbbda9ee-bedf-4eef-bdc4-cea94efcb6df.mp4

Changing the speed during playback (e.g. pressing [ then undo with ]) makes playback smooth again.

This also happens with other 60 fps videos, this one is just convenient. It is also possible to reproduce this by starting mpv with the url, but then you have to wait until the file is opened before reducing speed to 1 and starting playback, otherwise it doesn't happen. Does not happen when any of the configuration parameters are changed, with the exception that it also happens with --gpu-api=vulkan. Was tested on a 60Hz monitor and I don't have a monitor with a different refresh rate to test with.

Expected behavior

Smooth playback

Actual behavior

Jittery playback (reported jitter is till low though)

Log file

jitter.txt started with speed=2 smooth.txt started with speed=3 (for comparison)

Sample files

haasns 60 fps test video

[haasn]

~~I can't reproduce this with the command line options given. Playback is completely smooth for me with those options.~~

Edit: Nvm, my bad. I was testing incorrectly. I can reproduce now.

[christoph-heinrich]

I have managed to trigger this weird jitter during playback with changes in speed.

osd-msg1="jitter ${vsync-jitter}"
vo=gpu-next
video-sync=display-resample
video-sync-max-video-change=20
video-sync-max-factor=4
interpolation

Those specific values for video-sync-max-video-change and video-sync-max-factor aren't that important, as I have also had this happen with other values (even the default ones), but these seem to be particularly prone to triggering this for me. I'm displaying the reported jitter this time to show that it doesn't change when this happens.

Takes me about 10 seconds to trigger this in this video.

https://user-images.githubusercontent.com/8932183/182970703-6f895810-c13b-403f-85dc-0650f00d0fa9.mp4

I've also found another issue while testing this, but I'm pretty sure it's unrelated as it also happens with vo=gpu, so I'll open another issue about it after investigating a bit more.

[haasn]

So, what happens here is that, for some reason, the initialization with a different speed causes the mpv frame pts calculations to be 1) out-of-cadence (16 vs 17) with respect to the source frames, and 2) offset about halfway in between two source frames, for some reason.

1. on itself is sort of unavoidable and not a huge deal, but 2) is the game-breaker here. The solution needs to involve making mpv consistently pick PTS values that roughly align with source frames.

Perhaps one easy fix would be to, on a speed change, reset the "current PTS" to the true PTS of the previous frame?

[haasn]

The entire issue seems to stem from a single frame, actually. Here is the relative PTS jumps, starting with the very first frame after hitting 'space' to unpause:

8.666
16.334
16.667
16.333
16.667
16.999
17.001

As you can see, they are all spaced roughly correctly, except for the very first, which for some reason is only getting half the intended frame duration added. I suspect this may be because of some internal state of the mpv interpolation code not being correctly cleared when the speed changes.

This would explain why the issue only shows up for --speed=2, because that results in the exactly half-misaligned frames that trigger this bug. Something like --speed=3 would, presumably, result in one third misaligned frames. This also explains why slight speed changes can clear it, basically anything that shifts this vsync error by more than 1 ms takes the timestamps out of the "danger zone".

[Dudemanguy]

#13550 fixes this since it resets av values on speed changes.

[Dudemanguy]

Technically back now.