Issue with io/ffmpeg.py

[praateekmahajan]

RuntimeError: Traceback (most recent call last):
  File "/home/praateek/.local/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 40, in _worker_loop
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "<ipython-input-10-a53d73bbb9a6>", line 32, in __getitem__
    vid = skvideo.io.vread(self.__xs[index])
  File "/usr/local/lib/python2.7/dist-packages/skvideo/io/io.py", line 148, in vread
    for idx, frame in enumerate(reader.nextFrame()):
  File "/usr/local/lib/python2.7/dist-packages/skvideo/io/ffmpeg.py", line 293, in nextFrame
    yield self._readFrame()
  File "/usr/local/lib/python2.7/dist-packages/skvideo/io/ffmpeg.py", line 277, in _readFrame
    s = self._read_frame_data()
  File "/usr/local/lib/python2.7/dist-packages/skvideo/io/ffmpeg.py", line 271, in _read_frame_data
    raise RuntimeError("%s" % (err1,))
RuntimeError

So I am loading around 10,000 videos, and this issue seems to pop randomly. First I thought maybe it's one corrupted video files, but after having tried on different machines, I am sure the error just pops from any arbitary file.

        vid = skvideo.io.vread(filename)
        frames, width, height, channels = vid.shape
        time_steps = int(frames/self.timestep) #sample rate is number of frames to extract per second 
        return vid[::self.timestep]

I'm sharing my code just in case it helps figure it out, but I believe the behavior is random, because it does it load a lot of videos before throwing that error.

[beyondmetis]

Thanks for sharing. I know there is a problem like this when roughly half your system memory is used by python prior to a ffmpeg call.

How large are your largest videos?

[praateekmahajan]

Longest video is 71 seconds in length. All the videos have a frame rate of 25fps and resolution 320x240.

It is the UCF101 dataset if you want to look at it more. But yes, videos aren’t fancy neither corrupted

On 01-Dec-2017, at 3:38 AM, Todd Goodall [email protected] wrote:

Thanks for sharing. I know there is a problem like this when roughly half your system memory is used by python prior to a ffmpeg call.

How large are your largest videos?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[beyondmetis]

Hmm, strange. If it's not a specific video, and you are loading them one-at-a-time, then you shouldn't have an issue like this. FFmpeg is pretty stable. From the error, it seems like the FFmpeg process starts but then has a chance of not behaving correctly. Memory issues usually fail early at the ffprobe stage, so it's likely something else. I'm not sure what.

[n17r4m]

I am running into what may or may not be the same issue as this, #48, and #50. Perhaps ffmpeg has subtlely changed it's output format in a recent release? (I'm using ffmpeg 3.4 fwiiw).

I traced the problem to ffmpeg.py around line 270, namely,

arr = np.fromstring(self._proc.stdout.read(framesize), dtype=np.uint8)
assert len(arr) == framesize

len(arr) is 0, (on what I believe is the end of the video), and thus the assertion error is thrown, which is then rethrown as a RuntimeError.

As a dirty hack, changing

def nextFrame(self):
    for i in range(self.inputframenum):
        yield self._readFrame()

to

def nextFrame(self):
    for i in range(self.inputframenum - 1):
        yield self._readFrame()

(note the - 1) squelches the problem for me, but is probably not the correct fix.

[beyondmetis]

@n17r4m Thanks for catching what looks like a bug. I'll look into this.

[Lazare]

What we can do to mitigate this problem:

stop yielding as soon as we get an empty frame.

• benefit : no more RuntimeException in case we overestimate the number of frames
• drawback : getShape() won't always give you the right number of frames (it can overestimate it never underestimate)

I'll check if there's a better way to estimate the true number of frames than what's already done in the Readers, but I don't think so except by reading the video file entirely if the number of frames is not present in the metadata. I think whoever that in case of resample ("-r" option) the estimation can be improved, and I'm working on it.

[ajinkyaT]

Any updates on error?

[Lazare]

I don't have time to work on this project at the moment. I should be able to do it in August.

[fbenda]

I got the same issue while reading video files:

videogen = skvideo.io.vreader(args.input_video, outputdict=outpudict)
    for i, frame in enumerate(videogen):
        ...

my videos are 3840x1080 pixel and around 2-3 minutes long with 25fps, but it always crashes around the 20ft frame:

Traceback (most recent call last):
  File ".../cv35/lib/python3.5/site-packages/skvideo/io/ffmpeg.py", line 271, in _read_frame_data
    assert len(arr) == framesize
AssertionError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".../src/main.py", line 77, in <module>
    main(sys.argv[1:])
  File ".../src/main.py", line 65, in main
    for i, frame in enumerate(videogen):
  File ".../cv35/lib/python3.5/site-packages/skvideo/io/io.py", line 253, in vreader
    for frame in reader.nextFrame():
  File ".../cv35/lib/python3.5/site-packages/skvideo/io/ffmpeg.py", line 297, in nextFrame
    yield self._readFrame()
  File ".../cv35/lib/python3.5/site-packages/skvideo/io/ffmpeg.py", line 281, in _readFrame
    s = self._read_frame_data()
  File ".../cv35/lib/python3.5/site-packages/skvideo/io/ffmpeg.py", line 275, in _read_frame_data
    raise RuntimeError("%s" % (err1,))
RuntimeError

My outputdict looks like:

outpudict = {"-map": "0",
                 "-vcodec": "copy",
                 "-f": "mpegts"}

Also, if I remove the outputdict parameter it works without any exception for the full video. (almost 4000 frames)

[evanfitzking]

Hello,

Having this problem as well. [

Invalid UE golomb code
   Last message repeated 106 times
[matroska,webm @ 0x23d9540] Read error at pos. 3935349 (0x3c0c75)
len(ffprobe_result) 1802
Traceback (most recent call last):
 File "/home/hamsolo/.virtualenvs/hamsolo/lib/python3.5/site-packages/skvideo/io/ffmpeg.py", line 271, in _read_frame_data
   assert len(arr) == framesize
AssertionError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
 File "processNewClips.py", line 269, in <module>
   main()
 File "processNewClips.py", line 235, in main
   for (frame_number, frame_data, frame_time) in classifyVideoFrames(clip[1], args.jump):
 File "processNewClips.py", line 54, in classifyVideoFrames
   for frame in itertools.islice(videoReader, 0, None, jump):
 File "/home/hamsolo/.virtualenvs/hamsolo/lib/python3.5/site-packages/skvideo/io/io.py", line 253, in vreader
   for frame in reader.nextFrame():
 File "/home/hamsolo/.virtualenvs/hamsolo/lib/python3.5/site-packages/skvideo/io/ffmpeg.py", line 297, in nextFrame
   yield self._readFrame()
 File "/home/hamsolo/.virtualenvs/hamsolo/lib/python3.5/site-packages/skvideo/io/ffmpeg.py", line 281, in _readFrame
   s = self._read_frame_data()
 File "/home/hamsolo/.virtualenvs/hamsolo/lib/python3.5/site-packages/skvideo/io/ffmpeg.py", line 275, in _read_frame_data
   raise RuntimeError("%s" % (err1,))
RuntimeError](url)

[beyondmetis]

Since this error keeps happening, send me videos. I’ll debug and attempt to patch scikit-video once I can regularly reproduce the problem.

[husnejahan]

ValueError: No way to determine width or height from video. Need -s in inputdict. Consult documentation on I/O.

[LittleLittleTree]

Use "ffmpeg" command, try "ffmpeg -i vid1.avi vid2.avi" and use vid2.avi to replace vid1.avi. For me, it can solve this problem. May be there are some formats of videos that ffmpeg doesn't support.

[beyondmetis]

@LittleLittleTree If that command works, that means that ffmpeg supports loading the original video without any extra flags. This should work perfectly in scikit video. If that is not the case, I would like more information (with an example video file) so I can reproduce.

[bmmlover]

I solved this problem by specifying correct codec in inputdict in FFmpegReader (in my case 'hevc'): in_params = {'-vcodec': 'hevc'} reader = FFmpegReader(filename, inputdict=in_params)

You can see appropriate codec in skvideo.io.ffprobe(filename)['video']['@codec_tag_string'].

[tailin1009]

I Just need some help! When I use skvideo.io.FFmpegReader to read videos in a for loop. My memory is growing. I just want to release the resource after reading the video information!!!

[beyondmetis]

@tailin1009 The videos should be released when the handle to them goes out of scope on the next iteration of your for loop. Can you share some minimal code snippet with your problem?

[HudsonHuang]

Use "ffmpeg" command, try "ffmpeg -i vid1.avi vid2.avi" and use vid2.avi to replace vid1.avi. For me, it can solve this problem. May be there are some formats of videos that ffmpeg doesn't support.

It solved my question, the FFmpegReader class might doesn't support some format as ffmpeg do.

[NagabhushanSN95]

Use "ffmpeg" command, try "ffmpeg -i vid1.avi vid2.avi" and use vid2.avi to replace vid1.avi. For me, it can solve this problem. May be there are some formats of videos that ffmpeg doesn't support.

I was facing the same problem @LittleLittleTree 's solution worked for me too. Thanks!

[NagabhushanSN95]

I think I found the problem. In my system, in that particular conda environment, there was a python package ffmpeg installed. This python package always overrides default ffmpeg installed. So, scikit-video uses this python package instead of the default ffmpeg on the system. The command ffmpeg -i vid1.avi vid2.avi uses default ffmpeg and hence it works.

I uninstalled the ffmpeg python package. Now everything works well. So, I guess it is not a problem with scikit-video.

[ZDL-Git]

@NagabhushanSN95 scikit-video built-in ffmpeg

[NagabhushanSN95]

Even i thought so. But this is what I found.

[joshuasilva414]

I had this error and I fixed it by not specifying frame width and height when calling skvideo.io.vread(). I'm not sure why this worked.

[liangbingzhao]

still gets this error