Predict the next frame based on the previous 16 frames

[pratikk-bulani]

Hi,

I want to predict the (n+1)th frame using the prior 'n' frames on UCF101 dataset. So I executed the following command 'bash scripts/ucf101/save_videos.sh' keeping '--vid_len 17' assuming that the 17th frame is predicted using the prior 16 frames (as UCF101 was trained using 16 frames per clip). But this didn't happen.

The output video shape of fake and rec gives 17 frames. How shall I perform the next frame prediction?

Thanks

[16lemoing]

Hi!

Sorry for the late reply. If you wish to predict the 17th frame from the 16 preceding ones, you should do the following:

• put '--vid_len 17' as you did.
• set '--x_cond_len 1024' (since one image is represented by 64 tokens you need to condition on 16*64=1024 tokens if you want to predict from 16 conditioning frames)

You will get an output of 17 frames, the 16 first ones being the real context and the last one being the prediction.

Hope it helps!