cross-modal-retrieval

cross-modal-retrieval copied to clipboard

PyTorch code for Mining on Heterogeneous Manifolds for Zero-shot Cross-modal Image Retrieval

Reference:

• F. Yang, Zheng Wang, Jing Xiao, Shin'ichi Satoh, "Mining on Heterogeneous Manifolds for Zero-shot Cross-modal Image Retrieval", AAAI 2020.

Prepare

Download the SYSU-MM01 dataset and run make preprocess to prepared the files needed for training.

Training

We train two single-modal models as well as a cross-modal model. Train single-model by running make train, you may want to edit argument modal to train model for specific modality. Train cross-modal by running make train_cross. Set CUDA_VISIBLE_DEVICES according to your needs.

Testing

Use make test to evaluate. Set CUDA_VISIBLE_DEVICES according to your needs.

Models

You may find our trained models below.

• Model for thermal images
• Model for visible images
• Cross-modal model for both modalities