video2dataset icon indicating copy to clipboard operation
video2dataset copied to clipboard

Investigate fusing

Open rom1504 opened this issue 3 years ago • 2 comments

https://docs.google.com/document/d/1_TD2KQLkEegszq4Eip568fc6cWnh9h0Jqj4Lc88t9Y0/edit#bookmark=id.4dhg93pb66bc

The data reader, the data writer and the subsampler are meant to be independent components. They should be implemented independently to make it easy to test and benchmarks them. However for performance reason we may want to let underlying libraries (eg ffmpeg or yt-dl) handle multiple responsibility and hence we can introduced fused components. For example we may fuse multiple subsamplers into a single one, or we may even want to fuse a reader and a subsampler. These fused components will behave the same as using a composition of 2 independent components but may be faster.

DoD: one fused component has been implemented, and performance comparison was done

rom1504 avatar Nov 13 '22 20:11 rom1504

is this worth trying now ?

rom1504 avatar Apr 23 '23 20:04 rom1504

yeah would be a good time, most obvious fusing at this point would be creating the "FFmpegSubsampler" which can do:

  • fps
  • clipping
  • resolution

all together

we should also fuse workers

iejMac avatar Apr 24 '23 02:04 iejMac