Two Stream Attention Convolutional Neural Networks

Contributions

We provide code for Two Stream Action Detection in Keras and each respective extension. Much of the credit of this repo goes to @joaogantunes and @ruipimentelfigueiredo.

Prerequisites

• python >= 2.7
• numpy (pip install numpy)
• ffmpeg built with gstreamer (sudo apt install ffmpeg)
• OpenCV 2.X or 3.X (build with opencv-contrib and ffmpeg and CUDA support)
• Pillow (pip install Pillow)
• CUDA + cuDNN (we used 9 and 7.1, respectively)
• Tensorflow (for backend GPU) (pip install tensorflow-gpu)
• Keras >= 2.1.6 (pip install keras)
• Pandas (pip install pandas) and SciPy (pip install scipy) (because of dependencies)
• Maplotlib + Seaborn (pip install seaborn) for plots

• (Optional) To use Keras to draw models you need graphviz and pydot.

Data

• AVA -- For the AVA dataset due to computational power constraints and for quickly testing the architecture, we made our own split of the dataset called mini-AVA, you can download it here

Models

You can get our Keras models here, includes pre-trained Kinetics, UCF101 models and our AVA models pre-trained on both.

License

This project is licensed under the MIT License - see the LICENSE.md file for details