clapper
clapper copied to clipboard
Rafostar
Nvidia: VP9 video freezes when seeking
Hello. I am having an issue where the video output initially plays fine, but freezes when attempting to seek. If you don't try to seek, it works fine, but if you do, you get stuck on single frame while the underlying sound plays fine.
Does not happen to all videos, but to most videos that I download via yt-dlp from YouTube.
Output of mediainfo for the video
[user@fedora]$ mediainfo -f "Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film [aqz-KE-bpKQ].mkv"
General
Count : 332
Count of stream of this kind : 1
Kind of stream : General
Kind of stream : General
Stream identifier : 0
Unique ID : 196920577566673718651321122498339342798
Unique ID : 196920577566673718651321122498339342798 (0x9425861136D92AA2A6804B8E7BF309CE)
Count of video streams : 1
Count of audio streams : 1
Video_Format_List : VP9
Video_Format_WithHint_List : VP9
Codecs Video : VP9
Video_Language_List : English
Audio_Format_List : AAC LC
Audio_Format_WithHint_List : AAC LC
Audio codecs : AAC LC
Complete name : Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film [aqz-KE-bpKQ].mkv
File name extension : Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film [aqz-KE-bpKQ].mkv
File name : Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film [aqz-KE-bpKQ]
File extension : mkv
Format : Matroska
Format : Matroska
Format/Url : https://matroska.org/downloads/windows.html
Format/Extensions usually used : mkv mk3d mka mks
Commercial name : Matroska
Format version : Version 4
File size : 1558631754
File size : 1.45 GiB
File size : 1 GiB
File size : 1.5 GiB
File size : 1.45 GiB
File size : 1.452 GiB
Duration : 634624
Duration : 10 min 34 s
Duration : 10 min 34 s 624 ms
Duration : 10 min 34 s
Duration : 00:10:34.624
Duration : 00:10:34:34
Duration : 00:10:34.624 (00:10:34:34)
Overall bit rate : 19647940
Overall bit rate : 19.6 Mb/s
Frame rate : 60.000
Frame rate : 60.000 FPS
Frame count : 38074
IsStreamable : Yes
File last modification date : UTC 2022-04-09 08:21:04
File last modification date (local) : 2022-04-09 10:21:04
Writing application : Lavf59.16.100
Writing application : Lavf59.16.100
Writing library : Lavf59.16.100
Writing library : Lavf59.16.100
ErrorDetectionType : Per level 1
Video
Count : 381
Count of stream of this kind : 1
Kind of stream : Video
Kind of stream : Video
Stream identifier : 0
StreamOrder : 0
ID : 1
ID : 1
Unique ID : 8277725086776764659
Format : VP9
Format : VP9
Commercial name : VP9
Codec ID : V_VP9
Codec ID/Url : http://www.webmproject.org/
Duration : 634566.000000
Duration : 10 min 34 s
Duration : 10 min 34 s 566 ms
Duration : 10 min 34 s
Duration : 00:10:34.566
Duration : 00:10:34:34
Duration : 00:10:34.566 (00:10:34:34)
Width : 3840
Width : 3 840 pixels
Height : 2160
Height : 2 160 pixels
Pixel aspect ratio : 1.000
Display aspect ratio : 1.778
Display aspect ratio : 16:9
Frame rate mode : CFR
Frame rate mode : Constant
Frame rate : 60.000
Frame rate : 60.000 FPS
Frame count : 38074
Color space : YUV
Delay : 0
Delay : 00:00:00.000
Delay : 00:00:00:00
Delay : 00:00:00.000 (00:00:00:00)
Delay, origin : Container
Delay, origin : Container
Language : en
Language : English
Language : English
Language : en
Language : eng
Language : en
Default : Yes
Default : Yes
Forced : No
Forced : No
colour_description_present : Yes
colour_description_present_Source : Container
Color range : Limited
colour_range_Source : Container
Color primaries : BT.709
colour_primaries_Source : Container
Transfer characteristics : BT.709
transfer_characteristics_Source : Container
Matrix coefficients : BT.709
matrix_coefficients_Source : Container
Audio
Count : 286
Count of stream of this kind : 1
Kind of stream : Audio
Kind of stream : Audio
Stream identifier : 0
StreamOrder : 1
ID : 2
ID : 2
Unique ID : 3104901973177927948
Format : AAC
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Commercial name : AAC
Format_AdditionalFeatures : LC
Codec ID : A_AAC-2
Duration : 634624.000000
Duration : 10 min 34 s
Duration : 10 min 34 s 624 ms
Duration : 10 min 34 s
Duration : 00:10:34.624
Duration : 00:10:34.624
Channel(s) : 6
Channel(s) : 6 channels
Channel positions : Front: L C R, Side: L R, LFE
Channel positions : 3/2/0.1
Channel layout : C L R Ls Rs LFE
Samples per frame : 1024
Sampling rate : 48000
Sampling rate : 48.0 kHz
Samples count : 30461952
Frame rate : 46.875
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Compression mode : Lossy
Delay : 0
Delay : 00:00:00.000
Delay : 00:00:00.000
Delay, origin : Container
Delay, origin : Container
Delay relative to video : 0
Delay relative to video : 00:00:00.000
Delay relative to video : 00:00:00.000
Title : ISO Media file produced by Google Inc.
Default : Yes
Default : Yes
Forced : No
Forced : No
VENDOR_ID : [0][0][0][0]
Reproduction instructions
- Download yt-dlp.
- Download a video from YouTube. I used:
yt-dlp https://www.youtube.com/watch?v=aqz-KE-bpKQHere's my output:
user@fedora ~]$ cd Downloads/
[user@fedora Downloads]$ yt-dlp https://www.youtube.com/watch?v=aqz-KE-bpKQ
[youtube] aqz-KE-bpKQ: Downloading webpage
[youtube] aqz-KE-bpKQ: Downloading android player API JSON
[youtube] aqz-KE-bpKQ: Downloading player afeb58ff
[info] aqz-KE-bpKQ: Downloading 1 format(s): 315+258
WARNING: Requested formats are incompatible for merge and will be merged into mkv
[download] Destination: Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film [aqz-KE-bpKQ].f315.webm
[download] 100% of 1.42GiB in 00:47
[download] Destination: Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film [aqz-KE-bpKQ].f258.m4a
[download] 100% of 29.34MiB in 00:02
[Merger] Merging formats into "Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film [aqz-KE-bpKQ].mkv"
Deleting original file Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film [aqz-KE-bpKQ].f315.webm (pass -k to keep)
Deleting original file Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film [aqz-KE-bpKQ].f258.m4a (pass -k to keep)
- Open the resulting video. You should not be able to seek without freezing.
Clapper version info
[user@fedora]$ flatpak info com.github.rafostar.Clapper
Clapper - Simple and modern GNOME media player
ID: com.github.rafostar.Clapper
Ref: app/com.github.rafostar.Clapper/x86_64/stable
Arch: x86_64
Branch: stable
Version: 0.5.2
License: GPL-3.0-or-later
Origin: flathub
Collection: org.flathub.Stable
Installation: system
Installed: 83.6 MB
Runtime: org.gnome.Platform/x86_64/42
Sdk: org.gnome.Sdk/x86_64/42
Commit: 3b0e04f2f3cd8913f912722e235d55bda489192d0230665e435f87f56cc8ce26
Parent: af3d35b800a90f71d5410d404b373f4a5c2695d0bd0cee2dc743412a28f0832d
Subject: Update Clapper to 0.5.2 (dad16df1)
Date: 2022-06-24 09:44:44 +0000
@ak-42
Is this with all experimental features disabled (pipewire and playbin3)? They are known to cause this kinds of problems, especially pipewire.
Is this with all experimental features disabled (pipewire and playbin3)? They are known to cause this kinds of problems, especially pipewire.
Yes. I did not change any settings, but I am on Fedora 36, which does use pipewire (by default).
[user@fedora ~]$ ps -e | grep pipewire
10064 ? 00:00:05 pipewire
10144 ? 00:00:11 pipewire-pulse
[user@fedora ~]$ pactl info | grep "Server Name"
Server Name: PulseAudio (on PipeWire 0.3.56)
Fedora 36, which does use pipewire (by default).
That's should be fine. Just wanted to confirm state of our experimental options.
Please share a little more info about your setup: What GPU? Is this Wayland (Fedora default) or Xorg? Also what decoder is being used during playback (name can be seen in video track selection button, next to progress bar)?
Decoder is nvvp9dec. Session is X11. EDIT: I also tried Wayland, and it's the same. Nvidia driver is 515.57. I have PRIME graphics, but I am using the Nvidia GPU completely (not in hybrid mode).
neofetch
[user@fedora ~]$ neofetch
.',;::::;,'. user@fedora
.';:cccccccccccc:;,. -----------
.;cccccccccccccccccccccc;. OS: Fedora Linux 36 (Workstation Edition) x86_64
.:cccccccccccccccccccccccccc:. Host: 81LK IdeaPad L340-15IRH Gaming
.;ccccccccccccc;.:dddl:.;ccccccc;. Kernel: 5.18.13-200.fc36.x86_64
.:ccccccccccccc;OWMKOOXMWd;ccccccc:. Uptime: 3 hours, 48 mins
.:ccccccccccccc;KMMc;cc;xMMc:ccccccc:. Packages: 2533 (rpm), 64 (flatpak)
,cccccccccccccc;MMM.;cc;;WW::cccccccc, Shell: bash 5.1.16
:cccccccccccccc;MMM.;cccccccccccccccc: Resolution: 1920x1080, 1920x1080
:ccccccc;oxOOOo;MMM0OOk.;cccccccccccc: DE: GNOME 42.3.1
cccccc:0MMKxdd:;MMMkddc.;cccccccccccc; WM: Mutter
ccccc:XM0';cccc;MMM.;cccccccccccccccc' WM Theme: Adwaita
ccccc;MMo;ccccc;MMW.;ccccccccccccccc; Theme: Adwaita [GTK2/3]
ccccc;0MNc.ccc.xMMd:ccccccccccccccc; Icons: Tela-circle-dark [GTK2/3]
cccccc;dNMWXXXWM0::cccccccccccccc:, Terminal: gnome-terminal
cccccccc;.:odl:.;cccccccccccccc:,. CPU: Intel i5-9300H (8) @ 2.400GHz
:cccccccccccccccccccccccccccc:'. GPU: NVIDIA GeForce GTX 1050 3 GB Max-Q
.:cccccccccccccccccccccc:;,.. GPU: Intel CoffeeLake-H GT2 [UHD Graphics 630]
'::cccccccccccccc::;,. Memory: 3785MiB / 7725MiB
Decoder is nvvp9dec.
Please try with:
GST_PLUGIN_FEATURE_RANK=vp9dec:300 flatpak run com.github.rafostar.Clapper
Decoder in the popover should change to vp9dec. This should play with software decoding. I want to check if this is Nvidia specific problem.
GST_PLUGIN_FEATURE_RANK=vp9dec:300 flatpak run com.github.rafostar.Clapper
That works, seeking is now fine. At 4k 60fps, it noticeably struggles with the UI being slow, presumably due to software decoding, but it does work. At 1080p on my device, there's no noticeable slowdown, it all works fine.
but it does work
OK. So we know its nvcodec issue.
@ak-42 Could you also check if stateless decoder also has this issue?
GST_PLUGIN_FEATURE_RANK=nvvp9sldec:300 flatpak run com.github.rafostar.Clapper
If this decoder was available, it should say nvvp9sldec as decoder name in that popover (next to progress bar). Please check if nvvp9sldec (with sl in middle) is being used and if it also has this problem.
GST_PLUGIN_FEATURE_RANK=nvvp9sldec:300 flatpak run com.github.rafostar.Clapper
When I run this, the decoder is shown as nvvp9sldec and the seeking works without issues.
When I run this, the decoder is shown as nvvp9sldec and the seeking works without issues.
In this case, you should open Clapper preferences window -> Tweaks -> Plugin ranking -> nvcodec, enable rank override and set all sl elements to a higher rank then their non-sl counterparts (e.g. 300). Restart player for the changes to be applied. This should solve your problem by using sl variants by default.
As for non-sl, those are older GStreamer implementations that AFAIK are scheduled to be deprecated in favor of sl versions in future. If you still wish for the old ones to be fixed, you should report that bug on GStreamer gitlab (as nvcodec issue, with mention that both software and sl decoders do not have this issue). That's where nvcodec plugin is maintained, thus cannot be fixed here. Although, one thing to consider for me is if I should make them default ones a little early on before GStreamer does :thinking:
most videos that I download via yt-dlp from YouTube
This is slightly unrelated to this issue, but in case of Clapper Flatpak (that you use), you can just open YT video URI or simply drag and drop video image from browser to play it. We do not have options to set codec, max res yet through (1080p60 max at the moment).
Thanks for the explanation. I actually checked the same video with GNOME Video (Totem) and it has a very similar seeking issue, which I suspect might also be caused by the same problem. I'll make the necessary config changes, and I look forward to a future where sl decoders are the default.
As for watching YouTube, I'd love to do that, but I am too used to Sponsoblock. yt-dlp has it implemented as well. It might actually be a cool optional addition for you to consider adding to Clapper. There's already an mpv implementation that you could check out as a reference, should you be interested.
(Totem) and it has a very similar seeking issue
Like I said, its GStreamer nvcodec issue, so every app that uses it will have it. I do not know if anyone reported it to GStreamer gitlab already through.
Ideally change to sl by default should happen in GStreamer, but if we want to make sl default here earlier then GStreamer does, it would be nice if few other Nvidia users could test both non-sl and sl decoders and confirm that sl ones indeed already work better.
Nonetheless, manually editing elements ranks in Clapper preferences like I described earlier should workaround this problem.
This isn't an Nvidia exclusive bug though, I'm an AMD user and video playback freezes while seeking as well, both will hwaccel and without it. I was able to reproduce this on totem as well so I'm pretty sure this gotta be a gstreamer bug.
@axel358
This isn't an Nvidia exclusive bug though, I'm an AMD user
No, the bug reported in this issue was Nvidia exclusive. Where old implementation that relies more on the Nvidia binary driver does not work, while newer one does. If you have problem with/without vaapi, its different one. If you are trying to play web/youtube video (what I assume from your screenshots), you should use Flatpak version of Clapper instead (only this package can currently do this right). Otherwise for local videos you can consider switching to vah264dec instead of vaapidecodebin (unless you are on Xorg), for both better performance and to avoid known vaapidecodebin issues.
But without using vaapi i also get the freezing as shown on the other screenshot. It was a local video and yes I'm on Xorg
But without using vaapi i also get the freezing as shown on the other screenshot. It was a local video and yes I'm on Xorg
If its local video and happens with and without vaapi and also (as you mentioned earlier) with Totem, then you should probably report this on GStreamer gitlab. Nonetheless this bug here was Nvidia stateful implementation specific and either changing to Nvidia statless or software decoding fixes this issue, thus your one is a different GStreamer problem.
(Totem) and it has a very similar seeking issue
Like I said, its GStreamer
nvcodecissue, so every app that uses it will have it. I do not know if anyone reported it to GStreamer gitlab already through.Ideally change to
slby default should happen in GStreamer, but if we want to makesldefault here earlier then GStreamer does, it would be nice if few other Nvidia users could test both non-sl and sl decoders and confirm thatslones indeed already work better.Nonetheless, manually editing elements ranks in Clapper preferences like I described earlier should workaround this problem.
So, I have NVIDIA proprietary driver, clapper installed from flathub, Fedora Workstation 36.
The video is the following:
mediainfo test.webm
General
Complete name : test.webm
Format : WebM
Format version : Version 4
File size : 160 MiB
Duration : 15 min 23 s
Overall bit rate : 1 453 kb/s
Writing application : Lavf58.76.100
Writing library : Lavf58.76.100
Video
ID : 1
Format : VP9
Codec ID : V_VP9
Duration : 15 min 23 s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 25.000 FPS
Color space : YUV
Language : English
Default : Yes
Forced : No
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Audio
ID : 2
Format : Opus
Codec ID : A_OPUS
Duration : 15 min 23 s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Bit depth : 32 bits
Compression mode : Lossy
Language : English
Default : Yes
Forced : No
flatpak run com.github.rafostar.Clapper test.webm
Decoder: nvvp9dec.
Starts fine. Seeking has video stuck on the image I land into, while audio correctly moves forward.
Every time I seek I get the followin error in the prompt:
(com.github.rafostar.Clapper:2): GStreamer-CRITICAL **: 17:58:52.344: gst_segment_to_stream_time: assertion 'segment->format == format' failed
GST_PLUGIN_FEATURE_RANK=vp9dec:300 flatpak run com.github.rafostar.Clapper
Decoder: vp9dec Seeking works fine. But the UI gets stuck when I stop it.
GST_PLUGIN_FEATURE_RANK=nvvp9sldec:300 flatpak run com.github.rafostar.Clapper test.webm
Decoder: nvvp9sldec.
Seeking works fine.
All is fine.
The ranking is reversed? In a ranking the first is better than the second, no? Here it looks like the codec with the higher value of the number is chosen over the codec with the lower value of the number.
Changing the ranking to have nvvp9sldec as default works.
In all three cases, when I click on the camera to open the pop-up with the codec, the Alt+Tab GNOME shortcut to switch windows doesn't work anymore.
Here it looks like the codec with the higher value of the number is chosen over the codec with the lower value of the number.
Yes, element that plays the same format with higher value is chosen, how higher does not matter. This is how GStreamer is designed.
The ranking is reversed?
We try to use (in most cases at least) values as they are decided by GStreamer devs. The sl ones are considered new, so they did not set them higher then old "stable" ones yet. You can change these permanently to your own liking in Clapper preferences window. Otherwise, problems with only particular elements should be reported on GStreamer gitlab, since they are developed there.
In all three cases, when I click on the camera to open the pop-up with the codec, the Alt+Tab GNOME shortcut to switch windows doesn't work anymore.
This is a known GTK4 issue. Not related to Clapper. See: https://gitlab.gnome.org/GNOME/gtk/-/issues/4701.
We try to use (in most cases at least) values as they are decided by GStreamer devs. The sl ones are considered new, so they did not set them higher then old "stable" ones yet. You can change these permanently to your own liking in Clapper preferences window. Otherwise, problems with only particular elements should be reported on GStreamer gitlab, since they are developed there.
I understood that.
What I mean is: if you tell me that something is a "ranking", and you use the word "ranking", what I understand is that the one ranked first is better than the one ranked second, the second is better than the third, and so on.
So I would have expected the decoder ranked first (i.e. number 1) to be preferred over the decoder ranked second (e.g. number 2).
If GStreamer do the opposite, fine. Maybe I missed that in the doc.
I have an Intel CPU, having the same issue on all codecs.
This one is for Nvidia only. If it happens with different decoders, then the cause is likely different and we would need a separate issue. Upstream already has this one: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1477 Maybe that's what happens to you as well? If it happens only sometimes for you, make sure you have pipewire disabled in Clapper preferences.