Unclear error while using gpu context

[eawer]

I've compiled ffmpeg and decord using steps I provided here - https://github.com/dmlc/decord/issues/89#issuecomment-670334699. It works like it is supposed to with cpu context, but when I'm trying to use gpu context - I see this error message (I'm running it in jupyter inside docker):

DECORDErrorTraceback (most recent call last)
<ipython-input-15-72d4a8598568> in <module>
----> 1 vr = VideoReader("F33QOm9Iws0-0.mp4", ctx=gpu(1))

~/.local/lib/python3.6/site-packages/decord-0.4.0-py3.6.egg/decord/video_reader.py in __init__(self, uri, ctx, width, height, num_threads)
     43         else:
     44             self._handle = _CAPI_VideoReaderGetVideoReader(
---> 45                 uri, ctx.device_type, ctx.device_id, width, height, num_threads, 0)
     46         if self._handle is None:
     47             raise RuntimeError("Error reading " + uri + "...")

~/.local/lib/python3.6/site-packages/decord-0.4.0-py3.6.egg/decord/_ffi/_ctypes/function.py in __call__(self, *args)
    173         check_call(_LIB.DECORDFuncCall(
    174             self.handle, values, tcodes, ctypes.c_int(num_args),
--> 175             ctypes.byref(ret_val), ctypes.byref(ret_tcode)))
    176         _ = temp_args
    177         _ = args

~/.local/lib/python3.6/site-packages/decord-0.4.0-py3.6.egg/decord/_ffi/base.py in check_call(ret)
     61     """
     62     if ret != 0:
---> 63         raise DECORDError(py_str(_LIB.DECORDGetLastError()))
     64 
     65 

DECORDError: [22:29:43] /root/decord/src/video/nvcodec/cuda_parser.h:39: CUDAError: cuvidCreateVideoParser(&parser_, &parser_info_) failed with error: 

And this is what appears in docker logs:

[06:52:04] /root/decord/src/video/nvcodec/cuda_threaded_decoder.cc:36: Using device: GeForce GTX 1060 3GB
[06:52:04] /root/decord/src/video/nvcodec/cuda_threaded_decoder.cc:56: Kernel module version 450.51, so using our own stream.

The code I'm using is pretty simple:

vr = VideoReader("F33QOm9Iws0-0.mp4", ctx=gpu(1))

ffmpeg -version:

ffmpeg version N-98652-g7031a7b Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)
configuration: --prefix=/root/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/root/ffmpeg_build/include --extra-ldflags=-L/root/ffmpeg_build/lib --extra-libs='-lpthread -lm' --bindir=/root/bin --enable-cuda-nvcc --enable-cuvid --enable-nvenc --enable-nonfree --enable-libnpp --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64 --enable-gpl --enable-gnutls --enable-libaom --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree
libavutil      56. 58.100 / 56. 58.100
libavcodec     58. 99.100 / 58. 99.100
libavformat    58. 49.100 / 58. 49.100
libavdevice    58. 11.101 / 58. 11.101
libavfilter     7. 87.100 /  7. 87.100
libswscale      5.  8.100 /  5.  8.100
libswresample   3.  8.100 /  3.  8.100
libpostproc    55.  8.100 / 55.  8.100

nvidia-smi:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.51.05    Driver Version: 450.51.05    CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce GTX 106...  On   | 00000000:01:00.0 Off |                  N/A |
| 39%   58C    P0    40W / 120W |   1376MiB /  3019MiB |     35%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  GeForce GTX 106...  On   | 00000000:02:00.0 Off |                  N/A |
|  0%   48C    P8    10W / 120W |     12MiB /  3019MiB |      1%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   2  GeForce GTX 106...  On   | 00000000:04:00.0 Off |                  N/A |
|  0%   36C    P8     5W / 120W |     12MiB /  3019MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   3  GeForce GTX 106...  On   | 00000000:05:00.0 Off |                  N/A |
|  0%   40C    P8     4W / 120W |     12MiB /  3019MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

decord version (built ftom the latest master with cmake .. -DUSE_CUDA=ON) - 0.4.0

[willprice]

Seems to be the same issue as #16

[zhreshold]

do you happen to use the nvidia-docker and what's the version?

[eawer]

@zhreshold yes, all of nvcr.io/nvidia/pytorch:20.07-py3, nvcr.io/nvidia/pytorch:20.03-py3 and nvcr.io/nvidia/mxnet:20.07-py3 failed. The output I posted is for nvcr.io/nvidia/pytorch:20.07-py3

[eawer]

If you mean the version of the docker itself, it's 19.03.8, build afacb8b7f0 and 19.03.12, build 48a66213fe

[willprice]

@zhreshold I get the same error natively (not under docker)

[bravma]

I have the same issue when using nvidia/cuda:10.1-cudnn7-devel-ubuntu18.04