denseflow
denseflow copied to clipboard
GPU Memory error when running DenseFlow
Running denseflow using the mmaction 'rawframes.py' script works for the RGB extraction but fails when tryign to extract flow images, throwing the following error:
terminate called after throwing an instance of 'cv::Exception' what(): OpenCV(4.2.0) /home/******/opencv-4.2.0/modules/core/src/cuda_host_mem.cpp:95: error: (-217:Gpu API call) out of memory in function 'allocate'
I only use 1 gpu (eGPU), but set it as such when calling script:
python build_rawframes.py /mnt/d/Data/ava/videos_15min/ /mnt/d/Data/ava/rawframes/ --task flow --level 1 --mixed-ext --flow-type tvl1 --num-gpu 1 --resume
Any ideas?
~~See https://github.com/open-mmlab/denseflow/issues/18~~
Please post the full stacktraces
Try making this number smaller
https://github.com/open-mmlab/denseflow/blob/master/include/dense_flow.h#L95
I've tried to set the batch size to 32, but unfortunately to no avail. This is a bit of a differant beast when it comes to setup, as it is a WSL ubuntu VM on Win 10 bootcamp (Macbook Pro) using an Aurous RTX 2080ti 11Gb GPU. So it might be a result of such a daisy chain of connections, but it is a bit odd given that at 11Gb I would have though it would be sufficient. Or possibly does the DenseFlow script not run through the GPU?
Unfortunately I can't seem to get a full stack trace from error, as it never really kills the process / script, and terminal / PyCharm just prints out as below.
flow 36 bhlFavrh7WU.mp4 tvl1 done push frames gray, video_flow_idx 127, batch_size 32 frames_gray_queue full, waiting... "/mnt/d/Data/ava/videos_15min/3IOE-Q3UWdA.mp4", frames ≈ 30600 "/mnt/d/Data/ava/videos_15min/z3kgrh0L_80.mp4", frames ≈ 30600 "/mnt/d/Data/ava/videos_15min/_-Z6wFjXtGQ.mp4", frames ≈ 30600 push frames gray, video_flow_idx 0, batch_size 32 push frames gray, video_flow_idx 0, batch_size 32 push frames gray, video_flow_idx 0, batch_size 32 push frames gray, video_flow_idx 31, batch_size 32 push frames gray, video_flow_idx 31, batch_size 32 push frames gray, video_flow_idx 31, batch_size 32 push frames gray, video_flow_idx 63, batch_size 32 push frames gray, video_flow_idx 63, batch_size 32 push frames gray, video_flow_idx 63, batch_size 32 push frames gray, video_flow_idx 95, batch_size 32 push frames gray, video_flow_idx 95, batch_size 32 push frames gray, video_flow_idx 95, batch_size 32 terminate called after throwing an instance of 'cv::Exception' what(): OpenCV(4.2.0) /home/*****/opencv-4.2.0/modules/core/src/cuda_host_mem.cpp:95: error: (-217:Gpu API call) out of memory in function 'allocate'