memories icon indicating copy to clipboard operation
memories copied to clipboard
verified publisher icon pulsejet

go-vod: Hardware accelerated video playback stuttering extremely

Open Ra72xx opened this issue 1 year ago • 20 comments

Describe the bug 4K videos played back with go-vod from within the memories app are stuttering to the point of being unwatchable. HD videos work, but they are not completely fluent, either. Playing back the videos without hardware acceleration, from the Nextcloud files app or from a mounted WebDAV drive works fine (in the home network with no bandwith problem, i.e. probably no transcoding necessary, wondering why Memories/go-vod is transcoding at all). I tried several MKV and MP4 videos.

To Reproduce Click on a video from within the memories timeline and start the playback.

Platform:

  • OS: Ubuntu 23.10 / 24.04
  • Browser: Vivaldi flatpak, Firefox flatpak (stutters kind of differently from Vivaldi, regular smaller stutters)
  • Memories Version: current
  • Nextcloud Version: 28
  • PHP Version: 8.3

Additional context

Setup:

  • Podman containers for nginx, go-vod and php-fpm, all running in a common pod with common device mappings (e.g. for Nextcloud data and /dev/dri/renderD128)
  • php-fpm compiled from the official docker file with custom add-ons and ffmpeg (which should not be used for transcoding in this case, however, if I understand the instructions correctly)
  • Nextcloud data in a host directory, no external storage
  • Nginx reverse proxy configured according to the official instructions
  • Nextcloud served from a subdirectory (i.e. go-vod is started with --env NEXTCLOUD_HOST="https://host:port/nextcloud/" command line)
  • Memories uses a common basedir for all users
  • Memories setup page shows everything in green
  • The user php, the php-fpm pool and nginx are running under is in the video- and render-groups of the host

Memories entries in config.php:

  'memories.vod.path' => '/data/www/nextcloud/apps/memories/bin-ext/go-vod-amd64',
  'memories.index.path' => '/Multimedia/',
  'memories.vod.vaapi' => true,
  'memories.vod.bind' => 'localhost:47788',
  'memories.vod.connect' => 'localhost:47788',
  'memories.vod.ffmpeg' => '/usr/local/bin/ffmpeg',
  'memories.vod.ffprobe' => '/usr/local/bin/ffprobe',
  'memories.exiftool' => '/data/www/nextcloud/apps/memories/bin-ext/exiftool-amd64-glibc',
  'memories.db.triggers.fcu' => true,
  'memories.gis_type' => 2,
  'memories.vod.disable' => false,
  'memories.vod.qf' => 25,
  'memories.vod.external' => true,

Nothing special in the go-vod container logs. The line "Invalid URL:/" occurs very often, however, even without any video playback.

Mai 04 06:02:17 Obelix go-vod[1359033]: 2024/05/04 06:02:17 7mov6m6e4ri0-max: /usr/local/bin/ffmpeg -loglevel warning -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i "/data/storage/nextcloud_data/user/files/Multimedia/2024/20240417video.mkv" -copyts -fflags +genpts -vf "format=nv12|vaapi,hwupload,scale_vaapi=force_original_aspect_ratio=decrease:format=nv12" -map "0:v:0" "-c:v" h264_vaapi -global_quality 25 -map "0:a:0?" "-c:a" aac -ac 1 -start_number 0 -avoid_negative_ts disabled -f hls -hls_flags split_by_time -hls_time 3 -hls_segment_type mpegts -hls_segment_filename /tmp/go-vod/7mov6m6e4ri0-912933336/max-%06d.ts -force_key_frames "expr:gte(t,n_forced*3)" -
Mai 04 06:02:19 Obelix go-vod[1359033]: 2024/05/04 06:02:19 7mov6m6e4ri0-max: recv max-000000.ts
Mai 04 06:02:21 Obelix go-vod[1359033]: 2024/05/04 06:02:21 7mov6m6e4ri0-max: recv max-000001.ts
Mai 04 06:02:23 Obelix go-vod[1359033]: 2024/05/04 06:02:23 7mov6m6e4ri0-max: recv max-000002.ts
Mai 04 06:02:24 Obelix go-vod[1359033]: 2024/05/04 06:02:24 7mov6m6e4ri0-max: recv max-000003.ts
Mai 04 06:02:25 Obelix go-vod[1359033]: 2024/05/04 06:02:25 7mov6m6e4ri0-max: resuming transcoding
Mai 04 06:02:26 Obelix go-vod[1359033]: 2024/05/04 06:02:26 Invalid URL /
Mai 04 06:02:26 Obelix go-vod[1359033]: 2024/05/04 06:02:26 7mov6m6e4ri0-max: recv max-000004.ts
Mai 04 06:02:28 Obelix go-vod[1359033]: 2024/05/04 06:02:28 7mov6m6e4ri0-max: recv max-000005.ts
Mai 04 06:02:30 Obelix go-vod[1359033]: 2024/05/04 06:02:30 7mov6m6e4ri0-max: recv max-000006.ts
Mai 04 06:02:31 Obelix go-vod[1359033]: 2024/05/04 06:02:31 7mov6m6e4ri0-max: recv max-000007.ts
Mai 04 06:02:32 Obelix go-vod[1359033]: 2024/05/04 06:02:32 7mov6m6e4ri0-max: resuming transcoding
Mai 04 06:02:33 Obelix go-vod[1359033]: 2024/05/04 06:02:33 7mov6m6e4ri0-max: recv max-000008.ts
Mai 04 06:02:35 Obelix go-vod[1359033]: 2024/05/04 06:02:35 7mov6m6e4ri0-max: recv max-000009.ts
Mai 04 06:02:36 Obelix go-vod[1359033]: 2024/05/04 06:02:36 7mov6m6e4ri0-max: recv max-000010.ts
Mai 04 06:02:38 Obelix go-vod[1359033]: 2024/05/04 06:02:38 7mov6m6e4ri0-max: recv max-000011.ts
Mai 04 06:02:39 Obelix go-vod[1359033]: 2024/05/04 06:02:39 7mov6m6e4ri0-max: resuming transcoding
Mai 04 06:02:40 Obelix go-vod[1359033]: 2024/05/04 06:02:40 7mov6m6e4ri0-max: recv max-000012.ts
Mai 04 06:02:41 Obelix go-vod[1359033]: 2024/05/04 06:02:41 7mov6m6e4ri0-max: recv max-000013.ts
Mai 04 06:02:43 Obelix go-vod[1359033]: 2024/05/04 06:02:43 7mov6m6e4ri0-max: recv max-000014.ts
Mai 04 06:02:45 Obelix go-vod[1359033]: 2024/05/04 06:02:45 7mov6m6e4ri0-max: recv max-000015.ts
Mai 04 06:02:45 Obelix go-vod[1359033]: 2024/05/04 06:02:45 7mov6m6e4ri0-max: resuming transcoding
Mai 04 06:02:46 Obelix go-vod[1359033]: 2024/05/04 06:02:46 7mov6m6e4ri0-max: recv max-000016.ts
Mai 04 06:02:48 Obelix go-vod[1359033]: 2024/05/04 06:02:48 7mov6m6e4ri0-max: recv max-000017.ts
Mai 04 06:02:50 Obelix go-vod[1359033]: 2024/05/04 06:02:50 7mov6m6e4ri0-max: recv max-000018.ts
Mai 04 06:02:51 Obelix go-vod[1359033]: 2024/05/04 06:02:51 7mov6m6e4ri0-max: recv max-000019.ts
Mai 04 06:02:53 Obelix go-vod[1359033]: 2024/05/04 06:02:53 7mov6m6e4ri0-max: recv max-000020.ts
Mai 04 06:02:53 Obelix go-vod[1359033]: 2024/05/04 06:02:53 7mov6m6e4ri0-max: goal satisfied: 20
Mai 04 06:02:53 Obelix go-vod[1359033]: 2024/05/04 06:02:53 7mov6m6e4ri0-max: resuming transcoding
Mai 04 06:02:55 Obelix go-vod[1359033]: 2024/05/04 06:02:55 7mov6m6e4ri0-max: recv max-000021.ts
Mai 04 06:02:56 Obelix go-vod[1359033]: 2024/05/04 06:02:56 7mov6m6e4ri0-max: recv max-000022.ts
Mai 04 06:02:58 Obelix go-vod[1359033]: 2024/05/04 06:02:58 7mov6m6e4ri0-max: recv max-000023.ts
Mai 04 06:02:59 Obelix go-vod[1359033]: 2024/05/04 06:02:59 7mov6m6e4ri0-max: recv max-000024.ts
Mai 04 06:02:59 Obelix go-vod[1359033]: 2024/05/04 06:02:59 7mov6m6e4ri0-max: goal satisfied: 24

Ra72xx avatar May 04 '24 04:05 Ra72xx

What kind of hardware is this? (both client and server)

pulsejet avatar May 04 '24 04:05 pulsejet

Thanks for the quick reply. Server is a x86-64 with Intel N5105 and 16 GB, client is also an x86-64 with some core i5. Stuff is beefy enough for UHD media streaming, e.g. Emby also works on the server flawlessy with hardware acceleration. It's only go-vod that doesn't work.

Ra72xx avatar May 04 '24 04:05 Ra72xx

I investigated a bit further. It might indeed be a client (flatpak?) problem, as it does not occur with:

  • Vivaldi on Android
  • Memories app on Android
  • Vivaldi, Chrome, Firefox Desktop browsers (as flatpaks) as long as go-vod is not enabled

The problem occurs only with Vivaldi, Chrome and Firefox on my desktop installed as flatpaks. (I only have one desktop Linux computer, so I can't try out if it is a hardware thing.) So it might be some strange incompatibility from the output of go-vod with my hardware and/or flatpak clients.

Ra72xx avatar May 04 '24 05:05 Ra72xx

This is related to some browser weirdness but I haven't encountered this for a long time now.

  1. How does it stutter? Does the audio keep playing and video stutter? Does the loading spinner show up?
  2. If you switch streaming to a lower quality (e.g. 480p) does it still stutter?
  3. What happens on turning off hardware acceleration?
  4. Is this an HEVC video? Can you provide a sample video?
  5. Are the logs in the message above for a 4k video?

pulsejet avatar May 04 '24 11:05 pulsejet

  1. How does it stutter? Does the audio keep playing and video stutter? Does the loading spinner show up?
  • Vivaldi, Chrome: Video playback is interrupted by longer pauses (every few seconds), during which the spinner apperas.
  • Firefox: Not so long interruptions, but more of "micro stutter" every second or so, no spinner appears, seems a bit more "fluent".
  • Audio: Sometimes stopping with the video, sometimes continuing. No pattern discenible. Worse with the Chrome-based browsers.
  1. If you switch streaming to a lower quality (e.g. 480p) does it still stutter?

E.g. 720p works.

  1. What happens on turning off hardware acceleration?

Strange enough, videos aren't playing at all anymore in Memories when disabling HW acceleration. However, when I try to switch from external go-vod to internal go-vod, I get a permission error on the vaapi device, but then the videos play?!?

  1. Is this an HEVC video? Can you provide a sample video?

Should be H264..

  1. Are the logs in the message above for a 4k video?

Yes.

While checking this out, I made an interesting (puzzling?) observation. The home video video collection contains mostly MP4 from my smartphone and MKVs in which I joined several from those MP4 without any resampling with MKVToolNix.

Nextcloud files: Chrome-browsers play all those files without any problem. Firefox is only able to play the MP4s, claims "unsupported format" when clicking on the MKVs Memories with go-vod (external): Chrome browsers play all of those files, but stutter a lot Firefox playes all videos, both MP4 and MKV, stutters a bit less than Chrome.

And, as I said, everything is fine with mobile phones (and e.g. a Kodi client, which basically runs the same Ubuntu OS base as my laptop). I begin to think that this is some kind of strange codec/browser interaction specific to my laptop.

If you also think that this is no Memories problem, feel free to close the bug.

Ra72xx avatar May 04 '24 15:05 Ra72xx

It's very likely that the client browser can't keep up with 4K streaming; I've observed this on multiple devices. Since we do live transcoding, decoding these streams is not particularly efficient and a lot of processing happens in video.js too, creating a CPU bottleneck. On the other hand, it works correctly on the platforms that can better utilize hardware acceleration.

E.g. 720p works.

This right here is the indication that the client can't decode fast enough. You can try reducing the transcode quality from the admin panel, this generally makes the video easier to decode.

pulsejet avatar May 05 '24 06:05 pulsejet

It's very likely that the client browser can't keep up with 4K streaming;

The videos are stored rather high-quality MP4-encoded on the server and without go-vod in the middle they can be displayed without any problems on the same client - be it from a mounted WebDAV drive or even from within the NC files-view - with the same browser. As soon as the files are re-encoded with go-vod, they begin to stutter. Another observation: If the video is not re-encoded, the download rate displayed by Gnome system monitor on the client is constantly high. If, however, the video is re-encoded by go-vod, the download rate oscillates between many very low values, followed by incredible high-speed bursts. ffmpeg on the server, however, has a very low cpu usage, so I assume that basically the hardware encoding works, but the way the content is streamed to the client (and, as I said, only one client in my household is affected) has some issue? Does go-vod use some client-specific parameters when encoding?

Ra72xx avatar May 06 '24 04:05 Ra72xx

That's expected -- transcoded videos will always be way less efficient than the original. The point of adaptive streaming is not increasing efficiency; it's to reduce the size when bandwidth is insufficient or an unsupported codec (e.g. HEVC) is used by the original.

One straightforward workaround would be to use "Direct" quality; this streams the original video. Usually if that fails it will fall back to transcoding.

pulsejet avatar May 06 '24 05:05 pulsejet

Thanks for your hints. However, using "direct" transcoding doesn't change the situation, still choppy video. go-vod command line in "direct" is like this:

/usr/local/bin/ffmpeg -loglevel warning -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i "/data/storage/nextcloud_data/user/files/Multimedia/2024/20240417video.mp4" -copyts -fflags +genpts -vf "format=nv12|vaapi,hwupload,scale_vaapi=force_original_aspect_ratio=decrease:format=nv12" -map "0:v:0" "-c:v" h264_vaapi -global_quality 25 -map "0:a:0?" "-c:a" aac -ac 1 -start_number 0 -avoid_negative_ts disabled -f hls -hls_flags split_by_time -hls_time 3 -hls_segment_type mpegts -hls_segment_filename /tmp/go-vod/y1cz6aioklo0-1786502357/max-%06d.ts -force_key_frames "expr:gte(t,n_forced*3)" -

Ra72xx avatar May 06 '24 15:05 Ra72xx

"Direct" means "no transcoding" so not sure what's going on here

pulsejet avatar May 06 '24 16:05 pulsejet

I have the similar issue. It happens both on iphone (not so prominent, happens sometimes) and Desktop/server machine (very prominent, every 5-6 seconds there is a pause for 2-3 seconds, and then it resumes playback).

Server: i9-13900K transcoding using NVENC/CUDA on 4080 Super // Windows 11 Ent 22631.3447 // Docker 4.30.0

Client 1: same machine from above ^ Chrome 124.0.6367.119 Client 2: Safari on iphone 15ProMax / IOS 17.4.1

Network: Client1 is localhost, Client2 is connected trough Asus ET12 on Wifi 6, 1000/1000Mbps.

Below is the short log clipping during the playback of which problem occurs on Client 1.

2024-05-08 18:00:28 2024/05/08 18:00:28 dkbl6fr7y280-720p: /usr/local/bin/ffmpeg -loglevel warning -ss 36.000000 -hwaccel cuda -i "/mnt/ncdata/admin/files/Iphone 15 Pro Max/2024/24-05-04 20-14-35 4110.mov" -copyts -fflags +genpts -vf "format=nv12|cuda,hwupload,scale_cuda=force_original_aspect_ratio=decrease:passthrough=0:w=1280:h=1280" -map "0:v:0" "-c:v" h264_nvenc -preset p6 -tune ll -rc vbr -rc-lookahead 30 -cq 24 -map "0:a:0?" "-c:a" aac -ac 1 -start_number 12 -avoid_negative_ts disabled -f hls -hls_flags split_by_time -hls_time 3 -hls_segment_type mpegts -hls_segment_filename /tmp/go-vod/dkbl6fr7y280-79194972/720p-%06d.ts -force_key_frames "expr:gte(t,n_forced*3)" - 2024-05-08 18:00:29 2024/05/08 18:00:29 ffmpeg-error: [hevc @ 0x56189ece5f80] Multiple Dolby Vision RPUs found in one AU. Skipping previous. 2024-05-08 18:00:29 2024/05/08 18:00:29 ffmpeg-error: [hevc @ 0x56189eda87c0] Multiple Dolby Vision RPUs found in one AU. Skipping previous. 2024-05-08 18:00:29 2024/05/08 18:00:29 ffmpeg-error: [hevc @ 0x56189ee6b000] Multiple Dolby Vision RPUs found in one AU. Skipping previous. 2024-05-08 18:00:30 2024/05/08 18:00:30 dkbl6fr7y280-720p: recv 720p-000012.ts 2024-05-08 18:00:31 2024/05/08 18:00:31 dkbl6fr7y280-720p: recv 720p-000013.ts 2024-05-08 18:00:32 2024/05/08 18:00:32 dkbl6fr7y280-720p: recv 720p-000014.ts 2024-05-08 18:00:33 2024/05/08 18:00:33 dkbl6fr7y280-720p: recv 720p-000015.ts 2024-05-08 18:00:33 2024/05/08 18:00:33 dkbl6fr7y280-720p: recv 720p-000016.ts 2024-05-08 18:00:33 2024/05/08 18:00:33 dkbl6fr7y280-720p: resuming transcoding 2024-05-08 18:00:34 2024/05/08 18:00:34 dkbl6fr7y280-720p: recv 720p-000017.ts 2024-05-08 18:00:35 2024/05/08 18:00:35 dkbl6fr7y280-720p: recv 720p-000018.ts 2024-05-08 18:00:36 2024/05/08 18:00:36 dkbl6fr7y280-720p: recv 720p-000019.ts 2024-05-08 18:00:36 2024/05/08 18:00:36 dkbl6fr7y280-720p: recv 720p-000020.ts 2024-05-08 18:00:37 2024/05/08 18:00:37 dkbl6fr7y280-720p: resuming transcoding

atlas382 avatar May 08 '24 16:05 atlas382

I have the same problem. I am using a 4k video and have tried all the quality options. On direct and original the video plays fine because there is no transcoding. So my client, server and network can handle 4k video streaming. This bug only happens for me on the Auto quality option and 4k quality option. I am not sure why 4k is different from original or direct in this case.

The client is using hardware to decode the incoming stream and it's still stuttering. image

This is most likely due to the fact that the transcodes use too much bandwidth. This is my network usage when playing the video using the 4k quality setting. image

Here are the docker logs for the stream that stutters on auto quality. Shouldn't auto quality be downscaled to the size of the screen of the device that's playing the video? I don't see that happening here. It starts having problems around the time that says goal satisfied: 24 and never really recovers after that meaning the playback is never smooth and synced.

2024/05/15 11:35:10 8kzbrsc1fr40-max: /usr/local/bin/ffmpeg -loglevel warning -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -noautorotate -i /nextcloud/data/mrga/files/Photos/Memories/2024/05/20240512_144857.mp4 -copyts -fflags +genpts -vf "format=nv12|vaapi,hwupload,scale_vaapi=force_original_aspect_ratio=decrease:format=nv12" -map "0:v:0" "-c:v" h264_vaapi -global_quality 24 -map "0:a:0?" "-c:a" aac -ac 1 -start_number 0 -avoid_negative_ts disabled -f hls -hls_flags split_by_time -hls_time 3 -hls_segment_type mpegts -hls_segment_filename /tmp/go-vod/8kzbrsc1fr40-1704370960/max-%06d.ts -force_key_frames "expr:gte(t,n_forced*3)" -
2024/05/15 11:35:12 8kzbrsc1fr40-max: recv max-000000.ts
2024/05/15 11:35:13 8kzbrsc1fr40-max: recv max-000001.ts
2024/05/15 11:35:14 8kzbrsc1fr40-max: recv max-000002.ts
2024/05/15 11:35:15 8kzbrsc1fr40-max: recv max-000003.ts
2024/05/15 11:35:16 8kzbrsc1fr40-max: resuming transcoding
2024/05/15 11:35:17 8kzbrsc1fr40-max: recv max-000004.ts
2024/05/15 11:35:18 8kzbrsc1fr40-max: recv max-000005.ts
2024/05/15 11:35:19 8kzbrsc1fr40-max: recv max-000006.ts
2024/05/15 11:35:20 8kzbrsc1fr40-max: recv max-000007.ts
2024/05/15 11:35:21 8kzbrsc1fr40-max: resuming transcoding
2024/05/15 11:35:21 8kzbrsc1fr40-max: recv max-000008.ts
2024/05/15 11:35:23 8kzbrsc1fr40-max: recv max-000009.ts
2024/05/15 11:35:24 8kzbrsc1fr40-max: recv max-000010.ts
2024/05/15 11:35:25 8kzbrsc1fr40-max: recv max-000011.ts
2024/05/15 11:35:26 8kzbrsc1fr40-max: resuming transcoding
2024/05/15 11:35:26 8kzbrsc1fr40-max: recv max-000012.ts
2024/05/15 11:35:27 8kzbrsc1fr40-max: recv max-000013.ts
2024/05/15 11:35:29 8kzbrsc1fr40-max: recv max-000014.ts
2024/05/15 11:35:30 8kzbrsc1fr40-max: recv max-000015.ts
2024/05/15 11:35:30 8kzbrsc1fr40-max: resuming transcoding
2024/05/15 11:35:31 8kzbrsc1fr40-max: recv max-000016.ts
2024/05/15 11:35:32 8kzbrsc1fr40-max: recv max-000017.ts
2024/05/15 11:35:34 8kzbrsc1fr40-max: recv max-000018.ts
2024/05/15 11:35:35 8kzbrsc1fr40-max: recv max-000019.ts
2024/05/15 11:35:35 8kzbrsc1fr40-max: resuming transcoding
2024/05/15 11:35:36 8kzbrsc1fr40-max: recv max-000020.ts
2024/05/15 11:35:37 8kzbrsc1fr40-max: recv max-000021.ts
2024/05/15 11:35:39 8kzbrsc1fr40-max: recv max-000022.ts
2024/05/15 11:35:40 8kzbrsc1fr40-max: recv max-000023.ts
2024/05/15 11:35:41 8kzbrsc1fr40-max: recv max-000024.ts
2024/05/15 11:35:41 8kzbrsc1fr40-max: goal satisfied: 24
2024/05/15 11:35:47 8kzbrsc1fr40-max: resuming transcoding
2024/05/15 11:35:48 8kzbrsc1fr40-max: recv max-000025.ts
2024/05/15 11:35:49 8kzbrsc1fr40-max: recv max-000026.ts
2024/05/15 11:35:50 8kzbrsc1fr40-max: recv max-000027.ts
2024/05/15 11:35:51 8kzbrsc1fr40-max: recv max-000028.ts
2024/05/15 11:35:51 8kzbrsc1fr40-max: goal satisfied: 28
2024/05/15 11:35:57 t8h137o18i00-max: stopping stream
2024/05/15 11:35:57 t8h137o18i00-max: ffmpeg exited with status: -1
2024/05/15 11:36:04 8kzbrsc1fr40-max: resuming transcoding
2024/05/15 11:36:06 8kzbrsc1fr40-max: recv max-000029.ts
2024/05/15 11:36:07 8kzbrsc1fr40-max: recv max-000030.ts
2024/05/15 11:36:08 8kzbrsc1fr40-max: recv max-000031.ts
2024/05/15 11:36:09 8kzbrsc1fr40-max: recv max-000032.ts
2024/05/15 11:36:09 8kzbrsc1fr40-max: goal satisfied: 32
2024/05/15 11:36:11 8kzbrsc1fr40-max: resuming transcoding
2024/05/15 11:36:13 8kzbrsc1fr40-max: recv max-000033.ts
2024/05/15 11:36:14 8kzbrsc1fr40-max: recv max-000034.ts
2024/05/15 11:36:15 8kzbrsc1fr40-max: recv max-000035.ts
2024/05/15 11:36:16 8kzbrsc1fr40-max: recv max-000036.ts
2024/05/15 11:36:16 8kzbrsc1fr40-max: goal satisfied: 36
2024/05/15 11:36:19 8kzbrsc1fr40-max: resuming transcoding
2024/05/15 11:36:21 8kzbrsc1fr40-max: recv max-000037.ts
2024/05/15 11:36:22 8kzbrsc1fr40-max: recv max-000038.ts
2024/05/15 11:36:23 8kzbrsc1fr40-max: recv max-000039.ts
2024/05/15 11:36:24 8kzbrsc1fr40-max: recv max-000040.ts
2024/05/15 11:36:24 8kzbrsc1fr40-max: goal satisfied: 40

This is the output for 4k quality setting. This for some reason starts a transcode but it really shouldn't since the client can handle a direct file.

2024/05/15 11:39:21 7h4oo1y297g0-max: /usr/local/bin/ffmpeg -loglevel warning -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -noautorotate -i /nextcloud/data/mrga/files/Photos/Memories/2024/05/20240512_144857.mp4 -copyts -fflags +genpts -vf "format=nv12|vaapi,hwupload,scale_vaapi=force_original_aspect_ratio=decrease:format=nv12" -map "0:v:0" "-c:v" h264_vaapi -global_quality 24 -map "0:a:0?" "-c:a" aac -ac 1 -start_number 0 -avoid_negative_ts disabled -f hls -hls_flags split_by_time -hls_time 3 -hls_segment_type mpegts -hls_segment_filename /tmp/go-vod/7h4oo1y297g0-1704370960/max-%06d.ts -force_key_frames "expr:gte(t,n_forced*3)" -
2024/05/15 11:39:22 7h4oo1y297g0-max: recv max-000000.ts
2024/05/15 11:39:23 7h4oo1y297g0-max: recv max-000001.ts
2024/05/15 11:39:24 7h4oo1y297g0-max: recv max-000002.ts
2024/05/15 11:39:26 7h4oo1y297g0-max: recv max-000003.ts
2024/05/15 11:39:26 7h4oo1y297g0-max: resuming transcoding
2024/05/15 11:39:27 7h4oo1y297g0-max: recv max-000004.ts
2024/05/15 11:39:28 7h4oo1y297g0-max: recv max-000005.ts
2024/05/15 11:39:29 7h4oo1y297g0-max: recv max-000006.ts
2024/05/15 11:39:30 7h4oo1y297g0-max: recv max-000007.ts
2024/05/15 11:39:31 7h4oo1y297g0-max: resuming transcoding
2024/05/15 11:39:32 7h4oo1y297g0-max: recv max-000008.ts
2024/05/15 11:39:33 7h4oo1y297g0-max: recv max-000009.ts
2024/05/15 11:39:34 7h4oo1y297g0-max: recv max-000010.ts
2024/05/15 11:39:35 7h4oo1y297g0-max: recv max-000011.ts
2024/05/15 11:39:36 7h4oo1y297g0-max: resuming transcoding
2024/05/15 11:39:36 7h4oo1y297g0-max: recv max-000012.ts
2024/05/15 11:39:38 7h4oo1y297g0-max: recv max-000013.ts
2024/05/15 11:39:39 7h4oo1y297g0-max: recv max-000014.ts
2024/05/15 11:39:40 7h4oo1y297g0-max: recv max-000015.ts
2024/05/15 11:39:40 7h4oo1y297g0-max: resuming transcoding
2024/05/15 11:39:41 7h4oo1y297g0-max: recv max-000016.ts
2024/05/15 11:39:42 7h4oo1y297g0-max: recv max-000017.ts
2024/05/15 11:39:44 7h4oo1y297g0-max: recv max-000018.ts
2024/05/15 11:39:45 7h4oo1y297g0-max: recv max-000019.ts
2024/05/15 11:39:45 7h4oo1y297g0-max: resuming transcoding
2024/05/15 11:39:46 7h4oo1y297g0-max: recv max-000020.ts
2024/05/15 11:39:47 7h4oo1y297g0-max: recv max-000021.ts
2024/05/15 11:39:49 7h4oo1y297g0-max: recv max-000022.ts
2024/05/15 11:39:50 7h4oo1y297g0-max: recv max-000023.ts
2024/05/15 11:39:51 7h4oo1y297g0-max: recv max-000024.ts
2024/05/15 11:39:51 7h4oo1y297g0-max: goal satisfied: 24
2024/05/15 11:39:59 7h4oo1y297g0-max: resuming transcoding
2024/05/15 11:40:00 7h4oo1y297g0-max: recv max-000025.ts
2024/05/15 11:40:01 7h4oo1y297g0-max: recv max-000026.ts
2024/05/15 11:40:02 8kzbrsc1fr40-max: stopping stream
2024/05/15 11:40:02 8kzbrsc1fr40-max: ffmpeg exited with status: -1
2024/05/15 11:40:02 7h4oo1y297g0-max: recv max-000027.ts
2024/05/15 11:40:03 7h4oo1y297g0-max: recv max-000028.ts
2024/05/15 11:40:03 7h4oo1y297g0-max: goal satisfied: 28
2024/05/15 11:40:16 7h4oo1y297g0-max: resuming transcoding
2024/05/15 11:40:17 7h4oo1y297g0-max: recv max-000029.ts
2024/05/15 11:40:19 7h4oo1y297g0-max: recv max-000030.ts
2024/05/15 11:40:20 7h4oo1y297g0-max: recv max-000031.ts
2024/05/15 11:40:21 7h4oo1y297g0-max: recv max-000032.ts
2024/05/15 11:40:21 7h4oo1y297g0-max: goal satisfied: 32

Network usage for 1440p. image

Finally this is the 1440p transcode that does not stutter and anything below this resolution is also fine during playback.

2024/05/15 11:44:09 7h4oo1y297g0-1440p: /usr/local/bin/ffmpeg -loglevel warning -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -noautorotate -i /nextcloud/data/mrga/files/Photos/Memories/2024/05/20240512_144857.mp4 -copyts -fflags +genpts -vf "format=nv12|vaapi,hwupload,scale_vaapi=force_original_aspect_ratio=decrease:format=nv12:w=2560:h=2560" -map "0:v:0" "-c:v" h264_vaapi -global_quality 24 -map "0:a:0?" "-c:a" aac -ac 1 -start_number 0 -avoid_negative_ts disabled -f hls -hls_flags split_by_time -hls_time 3 -hls_segment_type mpegts -hls_segment_filename /tmp/go-vod/7h4oo1y297g0-1704370960/1440p-%06d.ts -force_key_frames "expr:gte(t,n_forced*3)" -
2024/05/15 11:44:10 7h4oo1y297g0-1440p: recv 1440p-000000.ts
2024/05/15 11:44:11 7h4oo1y297g0-1440p: recv 1440p-000001.ts
2024/05/15 11:44:11 7h4oo1y297g0-1440p: recv 1440p-000002.ts
2024/05/15 11:44:12 7h4oo1y297g0-1440p: recv 1440p-000003.ts
2024/05/15 11:44:12 7h4oo1y297g0-1440p: resuming transcoding
2024/05/15 11:44:13 7h4oo1y297g0-1440p: recv 1440p-000004.ts
2024/05/15 11:44:13 7h4oo1y297g0-1440p: recv 1440p-000005.ts
2024/05/15 11:44:14 7h4oo1y297g0-1440p: recv 1440p-000006.ts
2024/05/15 11:44:14 7h4oo1y297g0-1440p: recv 1440p-000007.ts
2024/05/15 11:44:15 7h4oo1y297g0-1440p: resuming transcoding
2024/05/15 11:44:15 7h4oo1y297g0-1440p: recv 1440p-000008.ts
2024/05/15 11:44:16 7h4oo1y297g0-1440p: recv 1440p-000009.ts
2024/05/15 11:44:16 7h4oo1y297g0-1440p: recv 1440p-000010.ts
2024/05/15 11:44:17 7h4oo1y297g0-1440p: recv 1440p-000011.ts
2024/05/15 11:44:17 7h4oo1y297g0-1440p: resuming transcoding
2024/05/15 11:44:18 7h4oo1y297g0-1440p: recv 1440p-000012.ts
2024/05/15 11:44:18 7h4oo1y297g0-1440p: recv 1440p-000013.ts
2024/05/15 11:44:19 7h4oo1y297g0-1440p: recv 1440p-000014.ts
2024/05/15 11:44:19 7h4oo1y297g0-1440p: recv 1440p-000015.ts
2024/05/15 11:44:20 7h4oo1y297g0-1440p: resuming transcoding
2024/05/15 11:44:20 7h4oo1y297g0-1440p: recv 1440p-000016.ts
2024/05/15 11:44:21 7h4oo1y297g0-1440p: recv 1440p-000017.ts
2024/05/15 11:44:21 7h4oo1y297g0-1440p: recv 1440p-000018.ts
2024/05/15 11:44:22 7h4oo1y297g0-1440p: recv 1440p-000019.ts
2024/05/15 11:44:23 7h4oo1y297g0-1440p: recv 1440p-000020.ts
2024/05/15 11:44:23 7h4oo1y297g0-1440p: goal satisfied: 20
2024/05/15 11:44:25 7h4oo1y297g0-1440p: resuming transcoding
2024/05/15 11:44:26 7h4oo1y297g0-1440p: recv 1440p-000021.ts
2024/05/15 11:44:27 7h4oo1y297g0-1440p: recv 1440p-000022.ts
2024/05/15 11:44:27 7h4oo1y297g0-1440p: recv 1440p-000023.ts
2024/05/15 11:44:28 7h4oo1y297g0-1440p: recv 1440p-000024.ts
2024/05/15 11:44:28 7h4oo1y297g0-1440p: goal satisfied: 24
2024/05/15 11:44:32 7h4oo1y297g0-1440p: resuming transcoding
2024/05/15 11:44:32 7h4oo1y297g0-1440p: recv 1440p-000025.ts
2024/05/15 11:44:33 7h4oo1y297g0-1440p: recv 1440p-000026.ts
2024/05/15 11:44:34 7h4oo1y297g0-1440p: recv 1440p-000027.ts
2024/05/15 11:44:34 7h4oo1y297g0-1440p: recv 1440p-000028.ts
2024/05/15 11:44:34 7h4oo1y297g0-1440p: goal satisfied: 28
2024/05/15 11:44:38 7h4oo1y297g0-1440p: resuming transcoding
2024/05/15 11:44:38 7h4oo1y297g0-1440p: recv 1440p-000029.ts
2024/05/15 11:44:39 7h4oo1y297g0-1440p: recv 1440p-000030.ts
2024/05/15 11:44:40 7h4oo1y297g0-1440p: recv 1440p-000031.ts
2024/05/15 11:44:40 7h4oo1y297g0-1440p: recv 1440p-000032.ts
2024/05/15 11:44:40 7h4oo1y297g0-1440p: goal satisfied: 32
2024/05/15 11:44:46 7h4oo1y297g0-1440p: resuming transcoding
2024/05/15 11:44:47 7h4oo1y297g0-1440p: recv 1440p-000033.ts
2024/05/15 11:44:48 7h4oo1y297g0-1440p: recv 1440p-000034.ts
2024/05/15 11:44:48 7h4oo1y297g0-1440p: recv 1440p-000035.ts
2024/05/15 11:44:49 7h4oo1y297g0-1440p: recv 1440p-000036.ts
2024/05/15 11:44:49 7h4oo1y297g0-1440p: goal satisfied: 36

And lastly network usage for a direct stream quality option: image

I think a quick fix can be applied here by adjusting the ffmpeg command to take into consideration a maximum allowed video bitrate since right now it's using too much bandwidth.

A full fix would adjust the transcode logic of the auto option. I don't understand how it works right now. At a minimum it should check client screen resolution and client network speed and adjust to that.

Tested on client: Browser: Chrome Version 124.0.6367.208 (Official Build) (64-bit) OS: Windows 11 Version 23H2 (OS Build 22631.3447) Client has hardware decoding for AVC, HEVC, AV1 etc.

smrganic avatar May 15 '24 12:05 smrganic

For now a global quality setting of 30 in the memories admin panel keeps my specific video using the 4k quality setting at around 70 Mbps. This makes no sense since at that point it would be better to serve the original video at 30 Mbps. But at least it's not stuttering anymore.

smrganic avatar May 15 '24 12:05 smrganic

For the same resolution and quality, the transcoded video will always be bigger than the original. This is not a bug (see https://github.com/pulsejet/memories/issues/1177#issuecomment-2095205553). To be simple, if direct works for you, use just direct (or lower the quality or streaming resolution)

Since the client decides what quality to play in the auto quality option, it might be possible to downgrade if there are stutters. This probably needs to be implemented in videojs though, since memories doesn't handle the streaming specifics.

pulsejet avatar May 15 '24 17:05 pulsejet

I don't find a sensible streaming setting with go-vod. UHD and sometimes even HD videos stutter, even when streaming in the local network. The same videos are also served from the some source with an Emby server, and here everything works without any special settings. Emby uses the following command line for the transcoding, if that helps (seemingly downscaling to HD): /app/emby/bin/ffmpeg -loglevel +timing -y -print_graphs_file "/config/logs/ffmpeg-transcode-9c2daafc-7686-41ee-a81d-ac905820ce83_1graph.txt" -copyts -start_at_zero -init_hw_device "vaapi=dev1:/dev/dri/renderD128" -init_hw_device qsv=qsvdev@dev1 -filter_hw_device qsvdev -f matroska,webm -c:v:0 h264 -threads:v:0 1 -hwaccel:v:0 vaapi -hwaccel_device:v:0 dev1 -hwaccel_output_format:v:0 vaapi -noautorotate -c:v:1 h264 -i "/media/example.mkv" -filter_complex "[0:0]hwmap@f1=mode=+read:derive_device=qsv,vpp_qsv@f2=width=1920:height=1080[f2_out0]" -map [f2_out0] -map 0:2 -sn -c:v:0 h264_qsv -b:v:0 3808002 -g:v:0 90 -maxrate:v:0 3808002 -bufsize:v:0 7616004 -keyint_min:v:0 90 -r:v:0 29.970029830932617 -profile:v:0 high -aud:v:0 1 -c:a:0 copy -disposition:a:0 default -max_delay 5000000 -avoid_negative_ts disabled -f segment -map_metadata -1 -map_chapters -1 -segment_format mpegts -segment_list "/config/transcoding-temp/5CA85B/5CA85B.m3u8" -segment_list_type m3u8 -segment_time 00:00:03.000 -segment_start_number 0 -individual_header_trailer 0 -write_header_trailer 0 -segment_write_temp 1 "/config/transcoding-temp/5CA85B/5CA85B_%d.ts"

Ra72xx avatar May 24 '24 14:05 Ra72xx

Are there any new developments in this case? I like Memories, but it is still unuasable for my home videos unfortunately :-( . In my LAN, I can watch the same videos flawlessly mounted with Gnome Online Accounts (WebDAV), from the Nextcloud Web interface and with Emby (serving the same Nextcloud folder). However, I still can't play most videos stutter free with Memories. I also do not really understand the logic of the corresponding settings - I can set local ffmpeg binaries, an external go-vod binary and the external go-vod container. These options don't seem to be really separated ui-wise, the quality settings are somewhere in between, and in the end it all ends up in ffmpeg doing the same transcoding, as far as I understand it?!

Image

Ra72xx avatar Apr 21 '25 05:04 Ra72xx

The solution for me was to adjust php limits for the user

I'm using nextcloud-aio on docker so your path might vary

Here's mine:

In DOCKER go to > nextcloud-aio-nextcloud > (exec -connect to container) > cd /var/www/html >

microdnf install nano nano .user.ini memory_limit = 4096M PHP_MEMORY_LIMIT = 4096M max_execution_time = 600 post_max_size = 20G upload_max_filesize = 120G Save and exit This solved the stutter problem when using browser.

(This next part applies to nextcloud-aio and nvidia gpu scenario) Prior to that you want to make sure that you have external go-vod container actually running, that it's attached to the nextcloud network... and for my usecase I had to assign it a static address by installing portainer, going into network settings, checking if its part of nextcloud-aio network, and after that manually changing it to 172.18.0.11:47788 (doesnt have to be the exact same but it has to obey the network portion of the address) , and than using the same address in memories admin config instead of localhost:47788 because for some reason it didn't work with just port.

atlas382 avatar Apr 21 '25 08:04 atlas382

The solution for me was to adjust php limits for the user

I'm using nextcloud-aio on docker so your path might vary

Here's mine:

In DOCKER go to > nextcloud-aio-nextcloud > (exec -connect to container) > cd /var/www/html >

microdnf install nano nano .user.ini memory_limit = 4096M PHP_MEMORY_LIMIT = 4096M max_execution_time = 600 post_max_size = 20G upload_max_filesize = 120G Save and exit This solved the stutter problem when using browser.

(This next part applies to nextcloud-aio and nvidia gpu scenario) Prior to that you want to make sure that you have external go-vod container actually running, that it's attached to the nextcloud network... and for my usecase I had to assign it a static address by installing portainer, going into network settings, checking if its part of nextcloud-aio network, and after that manually changing it to 172.18.0.11:47788 (doesnt have to be the exact same but it has to obey the network portion of the address) , and than using the same address in memories admin config instead of localhost:47788 because for some reason it didn't work with just port.

Thanks for the hint, but my php.ini already has quite similar limits, and that doesn't help, unfortunately.

Ra72xx avatar Apr 22 '25 14:04 Ra72xx

I've been seeing similar issues.

Running nextcloud bare metal on Ubuntu 24.04. Intel Arc B580 video card. It's been transcoding video fine with jellyfin. Have a dedicated SSD for the transcode temp dir. Files themselves stored on a ZFS partition (3x 8TB drives in raidz1). 64GB of ram in the system, up to 32 GB which can be consumed as part of the ZFS cache. AMD Ryzen 9 9900x cpu. Nextcloud/jellyfin server is behind an IIS reverse proxy (probably abnormal, but works with jellyfin transcodes)

Does this with any video I've tried. The source videos I've been testing are 4k 60fps videos recorded on my cell phone. Does this on local gigabit network (lan, not wan).

In Chrome, I get append errors and the video stops after a few seconds. Video plays smoothly until it stops. Same behavior in Edge. Image

In Firefox, the video is jittery and skips around. No error and doesn't outright stop, like in chrome.

Video seems to play fine from nextcloud in direct mode.

json file in transcode folder

{
    "qf": 25,
    "vaapi": true,
    "vaapiLowPower": false,
    "nvenc": false,
    "nvencTemporalAQ": false,
    "nvencScale": "cuda",
    "useTranspose": false,
    "forceSwTranspose": false,
    "useGopSize": false,
    "bind": "127.0.0.1:47788",
    "ffmpeg": "\/usr\/bin\/ffmpeg",
    "ffprobe": "\/usr\/bin\/ffprobe",
    "tempdir": "\/var\/transcodes\/nextcloud\/oc8dwe2inzba"
}

Log file from playback in chrome

2025/08/11 19:55:18 plfxatyeyh00: new manager for /zfs/data1/nextcloud/<user redacted>/files/Photos/Zoo 2025/PXL_20250718_153831126.mp42025/08/11 19:55:18 plfxatyeyh00-max: stopping stream
2025/08/11 19:55:18 plfxatyeyh00-max: /usr/bin/ffmpeg -loglevel warning -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 ->2025/08/11 19:55:20 plfxatyeyh00-max: recv max-000000.ts
2025/08/11 19:55:21 plfxatyeyh00-max: recv max-000001.ts
2025/08/11 19:55:22 plfxatyeyh00-max: recv max-000002.ts
2025/08/11 19:55:23 plfxatyeyh00-max: recv max-000003.ts
2025/08/11 19:55:24 plfxatyeyh00-max: recv max-000004.ts
2025/08/11 19:55:24 plfxatyeyh00-max: goal satisfied: 4
2025/08/11 19:56:23 plfxatyeyh00-max: stopping stream
2025/08/11 19:57:23 plfxatyeyh00: destroying manager
2025/08/11 19:57:23 plfxatyeyh00-max: stopping stream
2025/08/11 19:57:23 plfxatyeyh00-1440p: stopping stream
2025/08/11 19:57:23 plfxatyeyh00-480p: stopping stream
2025/08/11 19:57:23 plfxatyeyh00-1080p: stopping stream
2025/08/11 19:57:23 plfxatyeyh00-720p: stopping stream

I copied the ts files before they were deleted from the cache. Each file is about 2 seconds long. They are around 50MB each.

nivenfres avatar Aug 12 '25 00:08 nivenfres

I've been seeing similar issues.

Running nextcloud bare metal on Ubuntu 24.04. Intel Arc B580 video card. It's been transcoding video fine with jellyfin. Have a dedicated SSD for the transcode temp dir. Files themselves stored on a ZFS partition (3x 8TB drives in raidz1). 64GB of ram in the system, up to 32 GB which can be consumed as part of the ZFS cache. AMD Ryzen 9 9900x cpu. Nextcloud/jellyfin server is behind an IIS reverse proxy (probably abnormal, but works with jellyfin transcodes)

Does this with any video I've tried. The source videos I've been testing are 4k 60fps videos recorded on my cell phone. Does this on local gigabit network (lan, not wan).

In Chrome, I get append errors and the video stops after a few seconds. Video plays smoothly until it stops. Same behavior in Edge. Image

In Firefox, the video is jittery and skips around. No error and doesn't outright stop, like in chrome.

Video seems to play fine from nextcloud in direct mode.

json file in transcode folder

{
    "qf": 25,
    "vaapi": true,
    "vaapiLowPower": false,
    "nvenc": false,
    "nvencTemporalAQ": false,
    "nvencScale": "cuda",
    "useTranspose": false,
    "forceSwTranspose": false,
    "useGopSize": false,
    "bind": "127.0.0.1:47788",
    "ffmpeg": "\/usr\/bin\/ffmpeg",
    "ffprobe": "\/usr\/bin\/ffprobe",
    "tempdir": "\/var\/transcodes\/nextcloud\/oc8dwe2inzba"
}

Log file from playback in chrome

2025/08/11 19:55:18 plfxatyeyh00: new manager for /zfs/data1/nextcloud/<user redacted>/files/Photos/Zoo 2025/PXL_20250718_153831126.mp42025/08/11 19:55:18 plfxatyeyh00-max: stopping stream
2025/08/11 19:55:18 plfxatyeyh00-max: /usr/bin/ffmpeg -loglevel warning -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 ->2025/08/11 19:55:20 plfxatyeyh00-max: recv max-000000.ts
2025/08/11 19:55:21 plfxatyeyh00-max: recv max-000001.ts
2025/08/11 19:55:22 plfxatyeyh00-max: recv max-000002.ts
2025/08/11 19:55:23 plfxatyeyh00-max: recv max-000003.ts
2025/08/11 19:55:24 plfxatyeyh00-max: recv max-000004.ts
2025/08/11 19:55:24 plfxatyeyh00-max: goal satisfied: 4
2025/08/11 19:56:23 plfxatyeyh00-max: stopping stream
2025/08/11 19:57:23 plfxatyeyh00: destroying manager
2025/08/11 19:57:23 plfxatyeyh00-max: stopping stream
2025/08/11 19:57:23 plfxatyeyh00-1440p: stopping stream
2025/08/11 19:57:23 plfxatyeyh00-480p: stopping stream
2025/08/11 19:57:23 plfxatyeyh00-1080p: stopping stream
2025/08/11 19:57:23 plfxatyeyh00-720p: stopping stream

I copied the ts files before they were deleted from the cache. Each file is about 2 seconds long. They are around 50MB each.

I have got the same issue. Jellyfin works no issue with transcoding enabled. Memories stutters no matter the video I open.

marco-calautti avatar Aug 28 '25 09:08 marco-calautti