Scale Equivariance Improves Siamese Tracking

Ivan Sosnovik*, Artem Moskalev*, and Arnold Smeulders Scale Equivariance Improves Siamese Tracking, WACV 2021.

Introduction

Siamese trackers turn tracking into similarity estimation between a template and the candidate regions in the frame. Mathematically, one of the key ingredients of success of the similarity function is translation equivariance. Non-translation-equivariant architectures induce a positional bias during training, which hinders accurate localization. In real-life scenarios, however, the target undergos more transformations than just translation, e.g. rotation and scaling. In this work we focus on the later and demonstrate that extending Siamese tracking with built-in scale-equivariance improves tracking quality.

Results

Raw results and models are available here

Environment: The code is developed with Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz GPU: NVIDIA Titan RTX,

Requirements

To train models and to run evaluations, you need the following dependencies

yacs
scipy
shapely
opencv-python
numpy
pytorch
torchvision
pyyaml

You will need to install python API for MATLAB engine to run the VOT benchmarks.

Quick start

1. Download the weights for model initialization from here. We used SESiamFCResNet22_pretrained.pth. You can also run the script transfer_weights.py to generate your own weights.
2. Download the preprocessed training and testing datasets. Use the instructions provided here
3. Adjust the training config in configs/train.yaml according to your tasks.
4. Run the training script

CUDA_VISIBLE_DEVICES=0 python train_siamfc.py --cfg configs/train.yaml

5. To run the tracker from a snapshot, simply do the following

CUDA_VISIBLE_DEVICES=0 python test_siamfc.py \
    --checkpoint snapshot/SESiamFCResNet22/checkpoint_otb.pth \
    --dataset OTB2015 \
    --dataset_root ~/datasets/ \
    --cfg configs/test.yaml 

6. When the output is generated, you are able to evaluate the performance of the tracker. Use lib/core/eval_otb.py or lib/core/eval_vot.py. For example,

python lib/core/eval_otb.py \
    --dataset OTB2015 \
    --dataset_root ~/datasets \
    --result_path results \
    --tracker_reg "SE*" \

Acknowledgements

The Robert Bosch GmbH is acknowledged for financial support.

Citation

If you found this work useful in your research, please consider citing

@InProceedings{Sosnovik_2021_WACV,
    author    = {Sosnovik, Ivan and Moskalev, Artem and Smeulders, Arnold W.M.},
    title     = {Scale Equivariance Improves Siamese Tracking},
    booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
    month     = {January},
    year      = {2021},
    pages     = {2765-2774}
}

License

Licensed under an MIT license.