RSKP
RSKP copied to clipboard
LeonHLJ
The official implementation of 'Weakly Supervised Temporal Action Localization via Representative Snippet Knowledge Propagation' (CVPR 2022)
RSKP
Weakly Supervised Temporal Action Localization via Representative Snippet Knowledge Propagation (CVPR 2022)
Linjiang Huang (CUHK), Liang Wang (CASIA), Hongsheng Li (CUHK)
Overview
The experimental results on THUMOS14 are as below.
| Method \ mAP(%) | @0.1 | @0.2 | @0.3 | @0.4 | @0.5 | @0.6 | @0.7 | AVG |
|---|---|---|---|---|---|---|---|---|
| UntrimmedNet | 44.4 | 37.7 | 28.2 | 21.1 | 13.7 | - | - | - |
| STPN | 52.0 | 44.7 | 35.5 | 25.8 | 16.9 | 9.9 | 4.3 | 27.0 |
| W-TALC | 55.2 | 49.6 | 40.1 | 31.1 | 22.8 | - | 7.6 | - |
| AutoLoc | - | - | 35.8 | 29.0 | 21.2 | 13.4 | 5.8 | - |
| CleanNet | - | - | 37.0 | 30.9 | 23.9 | 13.9 | 7.1 | - |
| MAAN | 59.8 | 50.8 | 41.1 | 30.6 | 20.3 | 12.0 | 6.9 | 31.6 |
| CMCS | 57.4 | 50.8 | 41.2 | 32.1 | 23.1 | 15.0 | 7.0 | 32.4 |
| BM | 60.4 | 56.0 | 46.6 | 37.5 | 26.8 | 17.6 | 9.0 | 36.3 |
| RPN | 62.3 | 57.0 | 48.2 | 37.2 | 27.9 | 16.7 | 8.1 | 36.8 |
| DGAM | 60.0 | 54.2 | 46.8 | 38.2 | 28.8 | 19.8 | 11.4 | 37.0 |
| TSCN | 63.4 | 57.6 | 47.8 | 37.7 | 28.7 | 19.4 | 10.2 | 37.8 |
| EM-MIL | 59.1 | 52.7 | 45.5 | 36.8 | 30.5 | 22.7 | 16.4 | 37.7 |
| BaS-Net | 58.2 | 52.3 | 44.6 | 36.0 | 27.0 | 18.6 | 10.4 | 35.3 |
| A2CL-PT | 61.2 | 56.1 | 48.1 | 39.0 | 30.1 | 19.2 | 10.6 | 37.8 |
| ACM-BANet | 64.6 | 57.7 | 48.9 | 40.9 | 32.3 | 21.9 | 13.5 | 39.9 |
| HAM-Net | 65.4 | 59.0 | 50.3 | 41.1 | 31.0 | 20.7 | 11.1 | 39.8 |
| ACSNet | - | - | 51.4 | 42.7 | 32.4 | 22.0 | 11.7 | - |
| WUM | 67.5 | 61.2 | 52.3 | 43.4 | 33.7 | 22.9 | 12.1 | 41.9 |
| AUMN | 66.2 | 61.9 | 54.9 | 44.4 | 33.3 | 20.5 | 9.0 | 41.5 |
| CoLA | 66.2 | 59.5 | 51.5 | 41.9 | 32.2 | 22.0 | 13.1 | 40.9 |
| ASL | 67.0 | - | 51.8 | - | 31.1 | - | 11.4 | - |
| TS-PCA | 67.6 | 61.1 | 53.4 | 43.4 | 34.3 | 24.7 | 13.7 | 42.6 |
| UGCT | 69.2 | 62.9 | 55.5 | 46.5 | 35.9 | 23.8 | 11.4 | 43.6 |
| CO2-Net | 70.1 | 63.6 | 54.5 | 45.7 | 38.3 | 26.4 | 13.4 | 44.6 |
| D2-Net | 65.7 | 60.2 | 52.3 | 43.4 | 36.0 | - | - | - |
| FAC-Net | 67.6 | 62.1 | 52.6 | 44.3 | 33.4 | 22.5 | 12.7 | 42.2 |
| Ours | 71.3 | 65.3 | 55.8 | 47.6 | 38.2 | 25.4 | 12.5 | 45.1 |
Prerequisites
Recommended Environment
- Python 3.6
- Pytorch 1.5
- Tensorboard Logger
- CUDA 10.1
Note: Our code works with different PyTorch and CUDA versions, for high version of Pytorch, you need to change one line of our code according to this issue.
Data Preparation
-
Prepare THUMOS'14 dataset.
- We recommend using features and annotations provided by this repo.
-
Place the features and annotations inside a
dataset/Thumos14reduced/folder.
Usage
Training
You can easily train the model by running the provided script.
-
Refer to
options.py. Modify the argument ofdataset-rootto the path of yourdatasetfolder. -
Run the command below.
$ python main.py --run-type 0 --model-id 1
Models are saved in ./ckpt/dataset_name/model_id/
Evaulation
The trained model can be found here. (This saved model's result is slightly different from the one reported in our paper.)
Please put it into ./ckpt/dataset_name/model_id/.
- Run the command below.
$ python main.py --pretrained --run-type 1 --model-id 1 --load-epoch xxx
Please refer to the log in the same folder of saved models to set the load epoch of the best model.
Make sure you set the right model-id that corresponds to the model-id during training.
References
We referenced the repos below for the code.
Citation
@InProceedings{rskp,
title={Weakly Supervised Temporal Action Localization via Representative Snippet Knowledge Propagation},
author={Huang, Linjiang and Wang, Liang and Li, Hongsheng},
booktitle={IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2022}
}
Contact
If you have any question or comment, please contact the first author of the paper - Linjiang Huang ([email protected]).
LeonHLJ