Video file integrity checking not working

[timevortex]

Hey,

Just thought I'd report that video file integrity checking is not working with ffmpeg because ffmpeg-python does not throw any exception due to errors.

I played around to get this working and found using -loglevel + checking if anything was output to stderr works fairly well to detect problems. When using -err_detect I found that something is always output to stderr irrelevant of whether the file was OK or not (even when supplied with -nostats and -hide_banner). So I would need to parse that string if I wanted to figure out if there was actually an error. So I went with the easier -loglevel option that's working :)

Below is the quick hack I did to make things work. Not sure if you'd want to patch this in but here it is :) I can clean it up (or you can) if you want to patch it in.

index 6258496..db2d3c0 100755
--- a/check_mi.py
+++ b/check_mi.py
@@ -100,9 +100,8 @@ def arg_parser():
                                                                            '(psd, xcf,. and rare ones)',
                         dest='is_disable_extra')
     parser.add_argument('-x', '--err-detect', metavar='E', type=str,
-                        help='execute ffmpeg decoding with a specific err_detect flag %(metavar)s, \'strict\' is '
-                             'shortcut for +crccheck+bitstream+buffer+explode',
-                        dest='error_detect', default='default')
+                        help='execute ffmpeg decoding with a specific loglevel flag %(metavar)s',
+                        dest='error_detect', default='error')
     parser.add_argument('-l', '--strict_level', metavar='L', type=int,
                         help='uses different apporach for checking images depending on %(metavar)s integer value. '
                              'Accepted values 0,1 (default),2: 0 ImageMagick idenitfy, 1 Pillow library+ImageMagick, '
@@ -234,18 +233,18 @@ def is_target_file(filename):
     return file_ext in MEDIA_EXTENSIONS
 
 
-def ffmpeg_check(filename, error_detect='default', threads=0):
-    if error_detect == 'default':
-        stream = ffmpeg.input(filename)
-    else:
-        if error_detect == 'strict':
-            custom = '+crccheck+bitstream+buffer+explode'
-        else:
-            custom = error_detect
-        stream = ffmpeg.input(filename, **{'err_detect': custom, 'threads': threads})
-
+def ffmpeg_check(filename, error_detect='error', threads=0):
+#    if error_detect == 'default':
+#        stream = ffmpeg.input(filename)
+#    else:
+#        if error_detect == 'strict':
+#            custom = '+crccheck+bitstream+buffer+explode'
+#        else:
+    custom = error_detect
+    stream = ffmpeg.input(filename, **{'loglevel': custom, 'threads': threads})
     stream = stream.output('pipe:', format="null")
-    stream.run(capture_stdout=True, capture_stderr=True)
+    stdout, stderr = stream.run(capture_stdout=True, capture_stderr=True)
+    return stderr
 
 
 def save_csv(filename, data):
@@ -322,7 +321,9 @@ def check_file(filename, error_detect='default', strict_level=0, zero_detect=0,
                 magick_identify_check(filename)
 
         if file_ext in VIDEO_EXTENSIONS:
-            ffmpeg_check(filename, error_detect=error_detect, threads=ffmpeg_threads)
+            err = ffmpeg_check(filename, error_detect=error_detect, threads=ffmpeg_threads)
+            if err:
+                raise Exception("ffmpeg error: ", err)
 
     # except ffmpeg.Error as e:
     #     # print e.stderr

Background info

$ ffmpeg -version
ffmpeg version 4.2.4-1ubuntu0.1 Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 9 (Ubuntu 9.3.0-10ubuntu2)
configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil      56. 31.100 / 56. 31.100
libavcodec     58. 54.100 / 58. 54.100
libavformat    58. 29.100 / 58. 29.100
libavdevice    58.  8.100 / 58.  8.100
libavfilter     7. 57.100 /  7. 57.100
libavresample   4.  0.  0 /  4.  0.  0
libswscale      5.  5.100 /  5.  5.100
libswresample   3.  5.100 /  3.  5.100
libpostproc    55.  5.100 / 55.  5.100

$ pip3 list | egrep ffmpeg
ffmpeg-python          0.1.17 

[ftarlao]

Thank you for pointing out the issue (quite strange,...) I'll dive in this issue, in few weeks, my current latency is 4 months on average :-) Regards

[crass]

Latency is nearly 3 years and counting as of now :)

I've hit this issue as well. Here is an ffmpeg issue about err_detect not working for FLAC decoding. This is might be evidence that err_detect is not working in general, which appears to be the case in my experience with Ubuntu's ffmpeg 4.4.2 and also from git e70e9b655426 (20240301). Or its a red-herring ;)

I have a video file that produces error messages, but does not exit with failure when the -xerror option is used. When viewing there appear to be no corrupted frames, but gets stuck for a half a second at one spot. So I suspect there is corruption, but its not due to corrupted frames. So I think the solution above might create too many false positives for some. I've created a PR that allows for choosing to error when there is any error messages or to error only when corrupted frames are detected. See #21.