ActionMAE icon indicating copy to clipboard operation
ActionMAE copied to clipboard

verified publisher icon sangminwoo
→

Metadata

[AAAI 2023] Official pytorch implementation of "Towards Good Practices for Missing Modality Robust Action Recognition"

ActionMAE

Pytorch code for our AAAI 2023 paper "Towards Good Practices for Missing Modality Robust Action Recognition".

Action Recognition with Missing Modality

Missing Modality Action Recognition
Standard multi-modal action recognition assumes that the modalities used in the training stage are complete at inference time: (a) → (b). We address the action recognition problem in situations where such assumption is not established, i.e., when modalities are incomplete at inference time: (a) → (c). Our goal is to maintain performance in the absence of any input modality.

Get Started

$ git clone https://github.com/sangminwoo/ActionMAE.git
$ cd ActionMAE

Dependencies

  • Pytorch 1.11.0
  • CUDA Toolkit 11.3
  • NVIDIA Apex

Environment Setup

conda install pytorch==1.11.0 torchvision==0.12.0 torchaudio==0.11.0 cudatoolkit=11.3 -c pytorch

  • Goto NVIDIA Apex, and follow the instruction.

  • See requirements.txt for all python dependencies, and you can install them using the following command.

$ pip install -r requirements.txt

Train & Eval

$ ./train_val_actionmae_multigpu.sh

See/modify configurations in ActionMAE/lib/configs.py

Citation

@inproceedings{woo2023towards,
  title={Towards Good Practices for Missing Modality Robust Action Recognition},
  author={Woo, Sangmin and Lee, Sumin and Park, Yeonju and Nugroho, Muhammad Adi and Kim, Changick},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  volume={37},
  number={1},
  year={2023}
}

Acknowledgement

We appreciate much the nicely organized codes developed by MAE and pytorch-image-models. Our codebase is built on them.

← Metadata

16
Stars
1
Forks
Watchers

Owner

verified publisher icon sangminwoo

Metadata

[AAAI 2023] Official pytorch implementation of "Towards Good Practices for Missing Modality Robust Action Recognition"

Back