NVDEC/NVENC acceleration not applied

[cyrildtm]

Describe Your Environment

• Version of ZoneMinder 1.34.23
• How you installed ZoneMinder apt install zoneminder
• Full name and version of OS VM Host: Ubuntu 20.04/qemu VM Guest: Debian GNU/Linux 10 (buster) 5.10.0-0.bpo.3-amd64 PCI passthrough: Quadro P400
• Browser name and version (if this is an issue with the web interface) FireFox, latest

If the issue concerns a camera

• Make and Model
• frame rate
• resolution
• ZoneMinder Source Type: IP camera with rtsp unicast

Describe the bug A clear and concise description of what the bug is.

The passed-through Quadro P400 is working correctly in guest os: nvidia-smi lists the device when I manually type and run ffmpeg, nvidia-smi shows the process entry, and nvidia-smi dmon records a fluctuation in decoder & encoder usage. ffmpeg -vsync 0 -nostdin -err_detect aggressive -fflags discardcorrupt -hwaccel cuvid -c:v h264_cuvid -y -i 290466-video.mp4 -c:a copy -c:v h264_nvenc -movflags frag_keyframe+empty_moov+faststart test.mp4 and the output file plays just fine. Replacing the disk input file with the rtsp stream also works just fine.

But Zoneminder does not pick up my own build of ffmpeg with cuvid enabled.

In "Options -> Images":

OPT_FFMPEG [v] PATH_FFMPEG /usr/local/bin/ffmpeg FFMPEG_INPUT_OPTIONS -nostdin -err_detect aggressive -fflags discardcorrupt -vsync 0 -hwaccel cuvid -c:v h264_cuvid FFMPEG_OUTPUT_OPTIONS -c:v h264_nvenc -c:a copy -movflags frag_keyframe+empty_moov+faststart FFMPEG_FORMATS mp4* asf avi 3gp mp4**

In camera source "Source":

Options (empty) (tried hwaccel=cuvid,c:v=h264_cuvid but log said ffmpeg flags hwaccel and c:v were not recognized) DecoderHWAccelName cuvid DecoderHWAccelDevice (empty)

With the above setup, nvidia-smi does not report any usage from zma or zmc (empty process list). top shows same CPU usage as before the graphics card was installed.

Tried apt remove and apt install zoneminder. Tried reboot.

To Reproduce Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior A clear and concise description of what you expected to happen.

From the debug log, it seems that Zoneminder is not using the correct version of ffmpeg. It still says "drm" and no "cuvid". My own build has "cuvid" and but no "drm" in hwaccels.

Debug Logs

02/21/21 01:14:25.841095 zmc_m1[10931].DB1-zm_logger.cpp/255 [LogOpts: level=DB1 effective=DB1, screen=OFF, database=WAR, logfile=DB9->/var/log/zm/zmc_m1.log, syslog=WAR]
02/21/21 01:14:25.841288 zmc_m1[10931].DB1-zm_utils.cpp/269 [Detected a x86\x86-64 processor with AVX2]
02/21/21 01:14:25.841682 zmc_m1[10931].INF-zm_ffmpeg.cpp/82 [Enabling ffmpeg logs, as LOG_DEBUG+LOG_FFMPEG are enabled in options]
02/21/21 01:14:25.841722 zmc_m1[10931].DB1-zm_ffmpeg_camera.cpp/190 [ffmpegcamera: width 1920 height 1080 linesize 7680 colours 4 imagesize 8294400]
02/21/21 01:14:25.841747 zmc_m1[10931].DB1-zm_monitor.cpp/382 [encoder params # Lines beginning with # are a comment 
# For changing quality, use the crf option
# 1 is best, 51 is worst quality
#crf=18
#crf=22]
02/21/21 01:14:25.841776 zmc_m1[10931].DB1-zm_monitor.cpp/424 [monitor purpose=1]
02/21/21 01:14:25.841782 zmc_m1[10931].DB1-zm_monitor.cpp/438 [mem.size(8) SharedData=600 TriggerData=560 VideoStoreData=4128 timestamps=1920 images=120x8294400 = 995328000 total=995335272]
02/21/21 01:14:25.841994 zmc_m1[10931].DB1-zm_storage.cpp/84 [Loaded Storage area 2 'NewStorage']
02/21/21 01:14:25.842012 zmc_m1[10931].DB1-zm_monitor.cpp/442 [Storage path: /media/**]
02/21/21 01:14:25.842038 zmc_m1[10931].DB1-zm_monitor.cpp/604 [Unable to map file /dev/shm/zm.mmap.1 (995335272 bytes) to locked memory, trying unlocked]
02/21/21 01:14:25.842046 zmc_m1[10931].DB1-zm_monitor.cpp/607 [Mapped file /dev/shm/zm.mmap.1 (995335272 bytes) to unlocked memory]
02/21/21 01:14:26.310130 zmc_m1[10931].DB1-zm_monitor.cpp/517 [Monitor ** has function 3,
label format = '%N - %m/%d/%y %H:%M:%S', label X = 1, label Y = 1, label size = 2,
image buffer count = 120, warmup count = 5, pre-event count = 24, post-event count = 36, alarm frame count = 4,
fps report interval = 10, ref blend percentage = 12, alarm ref blend percentage = 12, track motion = 0]
02/21/21 01:14:26.310371 zmc_m1[10931].DB1-zm_zone.cpp/884 [Got 3 zones for monitor **]
02/21/21 01:14:26.328600 zmc_m1[10931].DB1-zm_monitor.cpp/2400 [Loaded monitor 1(**), 3 zones]
02/21/21 01:14:26.328651 zmc_m1[10931].INF-zmc.cpp/223 [Starting Capture version 1.34.23]
02/21/21 01:14:26.328901 zmc_m1[10931].DB1-zm_ffmpeg_camera.cpp/216 [Priming capture from rtsp://**]
02/21/21 01:14:26.328922 zmc_m1[10931].DB1-zm_ffmpeg_camera.cpp/379 [Calling avformat_open_input for rtsp://**]
02/21/21 01:14:27.280323 zmc_m1[10931].DB1-zm_ffmpeg_camera.cpp/472 [Failed to find decoder (h264_mmal)]
02/21/21 01:14:27.280428 zmc_m1[10931].DB1-zm_ffmpeg.cpp/335 [Dumping stream index i(0) index(0)]
02/21/21 01:14:27.280454 zmc_m1[10931].DB1-zm_ffmpeg.cpp/345 [    Stream #0:0]
02/21/21 01:14:27.280473 zmc_m1[10931].DB1-zm_ffmpeg.cpp/356 [frames:28, frame_size:0 stream timebase: 1/90000]
02/21/21 01:14:27.280494 zmc_m1[10931].DB1-zm_ffmpeg.cpp/359 [codec: h264]
02/21/21 01:14:27.280513 zmc_m1[10931].DB1-zm_ffmpeg.cpp/288 [90k stream tb numerator]
02/21/21 01:14:27.280538 zmc_m1[10931].DB1-zm_ffmpeg_camera.cpp/496 [vdpau]
02/21/21 01:14:27.280557 zmc_m1[10931].DB1-zm_ffmpeg_camera.cpp/496 [vaapi]
02/21/21 01:14:27.280574 zmc_m1[10931].DB1-zm_ffmpeg_camera.cpp/496 [drm]
02/21/21 01:14:27.280614 zmc_m1[10931].DB1-zm_ffmpeg_camera.cpp/501 [Device type cuvid is not supported.]
02/21/21 01:14:27.280634 zmc_m1[10931].DB1-zm_ffmpeg_camera.cpp/526 [decoder h264 hwConfig doesn't match our type: (null) != vaapi, pix_fmt vaapi_vld.]
02/21/21 01:14:27.280655 zmc_m1[10931].DB1-zm_ffmpeg_camera.cpp/526 [decoder h264 hwConfig doesn't match our type: (null) != vdpau, pix_fmt vdpau.]
02/21/21 01:14:27.280674 zmc_m1[10931].DB1-zm_ffmpeg_camera.cpp/512 [Decoder h264 does not support device type (null).]
02/21/21 01:14:27.280692 zmc_m1[10931].DB1-zm_ffmpeg_camera.cpp/548 [Failed to find suitable hw_pix_fmt.]
02/21/21 01:14:27.280847 zmc_m1[10931].DB1-zm_ffmpeg.cpp/330 [Dumping codec_context codec_type(0) codec_id(27 h264) width(1920) height(1080)  timebase(1/180000) format(yuv420p) gop_size 12 max_b_frames 0 me_cmp 0 me_range 0 qmin 2 qmax 31]
02/21/21 01:14:27.280875 zmc_m1[10931].DB1-zm_ffmpeg_camera.cpp/575 [HWACCEL not in use]
02/21/21 01:14:27.296256 zmc_m1[10931].DB1-zm_ffmpeg_camera.cpp/290 [raw frame from decoder: format 0 yuv420p 1920x1080 linesize:1920x960 pts: -9223372036854775808]
02/21/21 01:14:27.296287 zmc_m1[10931].DB1-zm_ffmpeg_camera.cpp/1086 [ffmpegcamera: width 1920 height 1080 linesize 7680 colours 4 imagesize 8294400]
02/21/21 01:14:27.296958 zmc_m1[10931].DB1-zm_ffmpeg_camera.cpp/1116 [Setup conversion context for 1920x1080 yuv420p to 1920x1080 rgba]
02/21/21 01:14:27.302090 zmc_m1[10931].INF-zm_monitor.cpp/2555 [Monitor-**: images:1 - Capturing at 1.00 fps, capturing bandwidth 32885bytes/sec]
02/21/21 01:14:27.311644 zmc_m1[10931].DB1-zm_ffmpeg_camera.cpp/290 [raw frame from decoder: format 0 yuv420p 1920x1080 linesize:1920x960 pts: 4118]
02/21/21 01:14:27.311682 zmc_m1[10931].DB1-zm_ffmpeg_camera.cpp/1086 [ffmpegcamera: width 1920 height 1080 linesize 7680 colours 4 imagesize 8294400]
02/21/21 01:14:27.322680 zmc_m1[10931].DB1-zm_ffmpeg_camera.cpp/290 [raw frame from decoder: format 0 yuv420p 1920x1080 linesize:1920x960 pts: 7429]
02/21/21 01:14:27.322702 zmc_m1[10931].DB1-zm_ffmpeg_camera.cpp/1086 [ffmpegcamera: width 1920 height 1080 linesize 7680 colours 4 imagesize 8294400]

[cyrildtm]

I tried compiling ffmpeg again with cuda and libavutil:

--incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --enable-libmp3lame --enable-libopenjpeg --enable-libx265 --enable-nvenc --enable-libx264 --enable-shared --enable-pic --enable-nonfree --enable-cuvid --enable-nvenc --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64

then uninstalled ffmpeg and libavutil from apt, which breaks zoneminder. Re-installed zoneminder and got it running.

In the end, I used sudo make install to install all ffmpeg components.

When Zoneminder is running, nvidia-smi dmon reports zero encoder&decoder usage.

$ ldd /usr/bin/zmc |grep 'libav'
        libavformat.so.58 => /usr/local/lib/libavformat.so.58 (0x00007ff71bdd5000)
        libavcodec.so.58 => /usr/local/lib/libavcodec.so.58 (0x00007ff71aadd000)
        libavutil.so.56 => /usr/local/lib/libavutil.so.56 (0x00007ff71a82b000)

$ whereis ffmpeg
ffmpeg: /usr/bin/ffmpeg /usr/local/bin/ffmpeg /usr/share/ffmpeg /usr/share/man/man1/ffmpeg.1.gz

$ whereis libavutil
libavutil: /usr/local/lib/libavutil.so /usr/local/lib/libavutil.a

$ which ffmpeg
/usr/local/bin/ffmpeg

$ ffmpeg
ffmpeg version N-101227-g7b30dad3a6 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 8 (Debian 8.3.0-6)
  configuration: --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --enable-libmp3lame --enable-libopenjpeg --enable-libx265 --enable-nvenc --enable-libx264 --enable-shared --enable-pic --enable-nonfree --enable-cuvid --enable-nvenc --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64
  libavutil      56. 66.100 / 56. 66.100
  libavcodec     58.125.100 / 58.125.100
  libavformat    58. 68.100 / 58. 68.100
  libavdevice    58. 12.100 / 58. 12.100
  libavfilter     7.107.100 /  7.107.100
  libswscale      5.  8.100 /  5.  8.100
  libswresample   3.  8.100 /  3.  8.100
  libpostproc    55.  8.100 / 55.  8.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'

~/ffmpeg-nvenc/ffmpeg-libav-cuda/ffmpeg/build_nvenc$ find . -regextype sed -regex '.*/libav[^so]*\.so'
./libavutil/libavutil.so
./libavfilter/libavfilter.so
./libavdevice/libavdevice.so

PATH_FFMPEG in the web GUI is always /usr/local/bin/ffmpeg

[cyrildtm]

Update: It seems that OPT_FFMPEG is only used by Filter. But it is not what I want. Reasons:

1. It does not create videos real-time. The best I can do with filters is to search the non-archived events and create videos for them. I will have to wait for the filter to be re-executed in the background, and so there will be a delay. Changing the re-run interval to a very short time like 1 second will run the entire Filter list too frequently.
2. The Monitor -> Storage configuration already covers video creation. I can choose from X264 encode and H264 passthrough. Why not adding an option to use external ffmpeg? Why doing it elsewhere making it more confusing?
3. I still cannot decode with NVDEC. In Monitor -> General I can define the source type as ffmpeg, but it is actually a pre-built library unless I build Zone Minder by myself. It is not external ffmpeg as I would think of when I see the option in Options -> Images -> OPT_FFMPEG. My CPU load is still crazy like 100% on one core at 30fps. Again, I cannot use Intel or AMD acceleration because I'm using Zone Minder in a virtual machine, and Intel CPU acceleration does not work in QEMU. Setting hwaccel to qsv or anything will generate an error saying hardware device is invalid, and video decoder falls back to software.

[cyrildtm]

Update: I put cuda into Source -> Source -> DecoderHWAccelName and it worked. Now nvidia-smi dmon reports decoding usage. For some reason, this did not work before, and since the last time I tried, I rebooted the system a few times during system upgrades. I will continue to monitor the encoding&decoding part to see if anything changes.

For encoding, the "Optional Encoder Parameters" in Source -> Storage does not allow hwaccel or c:v. It looks like only these options listed here are allowed: https://github.com/PlatformLab/x264/blob/7d0ff22e8c96de126be9d3de4952edd6d1b75a8c/common/base.c

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.