video_features icon indicating copy to clipboard operation
video_features copied to clipboard

A suggestion for speed up the processing

Open leftthomas opened this issue 3 years ago • 3 comments

I found that the code will only processing one video on each GPU, and used about 2.5G memory, the memory usage is slow for GPUs which have lager memory. I wonder if there is a way like batch size or multithreading processing, and full use the GPU memory, also save the time to processing the dataset.

leftthomas avatar Dec 23 '21 15:12 leftthomas

Well, it is often not useful as the utilization is likely already 100%. Anyway, you can achieve what you want just by running the same cli command with a different set videos. However, I see your point and such customization is useful but could be difficult to implement.

Some feature extractors support batching such as resnet.

v-iashin avatar Dec 23 '21 21:12 v-iashin

ok, thanks, I wonder why the conda env named torch_zoo but not raft for RAFT, it's confusing. And I also found an inconsistent problem for extracting features for THUMOS14 dataset (20 classes, temporal action version), I set the following parameters:

feature_type=i3d
flow_type=raft 
on_extraction=save_numpy
stack_size=16
step_size=16
extraction_fps=25

the extracted features of around 40 videos have less 1 segment compared with I3D code. And the data type of saved numpy is float64, but not float32.

leftthomas avatar Dec 24 '21 03:12 leftthomas

It is because torch_zoo supports multiple models, not only raft. It can be confusing for i3d with the raft backend but it should be ok for the rest.

Maybe different versions of ffmpeg is the problem. Can you compare the output length with the time? Does it produce meaningful result?

you can transform your numpy array to have float32 precision by calling .float() on it.

Please create separate issues for separate questions. So, other people could benefit.

v-iashin avatar Dec 24 '21 04:12 v-iashin

It should be supported now. One may start several processes on with a single GPU.

v-iashin avatar Aug 25 '22 07:08 v-iashin