LAEO-Net++

Official support code for LAEO-Net++ paper (IEEE TPAMI, 2021).

See previous version here

Quick start

The following demo predicts the LAEO label on a pair of heads included in subdirectory data/ava_val_crop. You can choose either to use a model trained on UCO-LAEO or a model trained on AVA-LAEO.

cd laeonetplus
python mains/ln_demo_test.py

Training code will be available soon.

LAEO prediction on a given input video file

Try this Google Colab to run LAEO prediction on a target video file: notebook

Detecting and tracking heads

Try this Google Colab to perform head detection and tracking on a target video file: notebook

Install

Clone this repository:

git clone [email protected]:AVAuco/laeonetplus.git

See library dependencies here. In addition, a requirements.txt file is provided.

Training (in progress)

We provide an example of training on AVA-LAEO data (using preprocessed samples): see the training document.

References

Marin-Jimenez, M. J., Kalogeiton, V., Medina-Suarez, P., Zisserman, A. (2021). LAEO-Net++: revisiting people Looking At Each Other in videos. IEEE transactions on Pattern Analysis and Machine Intelligence, PP, 10.1109/TPAMI.2020.3048482. Advance online publication. https://doi.org/10.1109/TPAMI.2020.3048482

@article{marin21pami,
  author    = {Mar\'in-Jim\'enez, Manuel J. and Kalogeiton, Vicky and Medina-Su\'arez, Pablo and and Zisserman, Andrew},
  title     = {{LAEO-Net++}: revisiting people {Looking At Each Other} in videos},
  journal   = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  doi       = {10.1109/TPAMI.2020.3048482},
  year      = {2021}
}

Conference version:

@inproceedings{marin19cvpr,
  author    = {Mar\'in-Jim\'enez, Manuel J. and Kalogeiton, Vicky and Medina-Su\'arez, Pablo and and Zisserman, Andrew},
  title     = {{LAEO-Net}: revisiting people {Looking At Each Other} in videos},
  booktitle = CVPR,
  year      = {2019}
}