Video-Prediction-using-PyTorch
Video-Prediction-using-PyTorch copied to clipboard
Video Predicting using ConvLSTM and pytorch
Video-Prediction-using-PyTorch
Repository for frame prediction on the MovingMNIST dataset using seq2seq ConvLSTM following either of these guides:
Libraries
Make sure you have the following libraries installed!
python=3.6.8
torch=1.1.0
torchvision=0.3.0
pytorch-lightning=0.7.1
matplotlib=3.1.3
tensorboard=1.15.0a20190708
Getting started
-
Install the above libraries
-
Clone this repo
git clone https://github.com/holmdk/Video-Prediction-using-PyTorch.git
cd ./Video-Prediction-using-PyTorch
- Run main.py
python main.py
- Navigate to http://localhost:6006/ for visualizing results
Results
The first row displays our predictions, the second row the ground truth and the third row the absolute error on a pixel-level. The first 8 columns are the input, followed by output in the final 8 columns. This matches the output from the Tensorboard logging.
After some iterations, we notice that our model is actually generating images of all zeros! This is a common issue people using ConvLSTM reports, however, do not be discouraged! Simply keep training the model, and you should start to see actual and plausible future predictions.
Initial results (500 steps)
After half an epoch (2500 steps)
Now, we are actually starting to see actual predictions, however blurry they might be.
Todo:
- [ ] Add video of predictions by model
- [ ] Implement other video prediction methods (feel free to contribute!)
- [ ] SVG
- [ ] PredRNN+
- [ ] E3D
- [ ] MIM