PATNet
PATNet copied to clipboard
Official PyTorch Implementation of Cross-Domain Few-Shot Semantic Segmentation, ECCV 2022
Cross-Domain Few-Shot Semantic Segmentation (CD-FSS)
This is the implementation of the paper "Cross-Domain Few-Shot Semantic Segmentation". For more information, check out the [paper] and [supp].
Introduction
The Cross-Domain Few-Shot Semantic Segmentation includes data from the Deepglobe [1], ISIC2018 [2-3], Chest X-ray [4-5], and FSS-1000 [6] datasets, which covers satellite images, dermoscopic images of skin lesions, X-ray images, and daily objects respectively. The selected datasets reflect real-world use cases for few-shot learning since collecting enough examples from above domains is often difficult, expensive, or in some cases not possible.
Datasets
The following datasets are used for evaluation in CD-FSS:
Source domain:
-
PASCAL VOC2012:
Download PASCAL VOC2012 devkit (train/val data):
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar
Download PASCAL VOC2012 SDS extended mask annotations from [Google Drive].
Target domains:
-
Deepglobe:
Home: http://deepglobe.org/
Direct: https://www.kaggle.com/datasets/balraj98/deepglobe-land-cover-classification-dataset
Preprocessed Data: https://drive.google.com/file/d/10qsi1NRyFKFyoIq1gAKDab6xkbE0Vc74/view?usp=sharing
-
ISIC2018:
Home: http://challenge2018.isic-archive.com
Direct (must login): https://challenge.isic-archive.com/data#2018
Class Information: data/isic/class_id.csv
-
Chest X-ray:
Home: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4256233/
Direct: https://www.kaggle.com/datasets/nikhilpandey360/chest-xray-masks-and-labels
-
FSS-1000:
Home: https://github.com/HKUSTCV/FSS-1000
Direct: https://drive.google.com/file/d/16TgqOeI_0P41Eh3jWQlxlRXG9KIqtMgI/view
Requirements
- Python 3.7
- PyTorch 1.5.1
- cuda 10.1
- tensorboard 1.14
Conda environment settings:
conda create -n patnet python=3.7
conda activate patnet
conda install pytorch=1.5.1 torchvision cudatoolkit=10.1 -c pytorch
conda install -c conda-forge tensorflow
pip install tensorboardX
Training
PASCAL VOC
python train.py --backbone {vgg16, resnet50} --fold 4 --benchmark pascal --lr 1e-3 --bsz 20 --logpath "your_experiment_name"
Testing
1. Deepglobe
python test.py --backbone {vgg16, resnet50} --benchmark deepglobe --nshot {1, 5} --load "path_to_trained_model/best_model.pt"
2. ISIC
python test.py --backbone {vgg16, resnet50} --benchmark isic --nshot {1, 5} --load "path_to_trained_model/best_model.pt"
3. Chest X-ray
python test.py --backbone {vgg16, resnet50} --benchmark lung --nshot {1, 5} --load "path_to_trained_model/best_model.pt"
4. FSS-1000
python test.py --backbone {vgg16, resnet50} --benchmark fss --nshot {1, 5} --load "path_to_trained_model/best_model.pt"
Citation
If you use this code for your research, please consider citing:
@inproceedings{lei2022cross,
title={Cross-Domain Few-Shot Semantic Segmentation},
author={Lei, Shuo and Zhang, Xuchao and He, Jianfeng and Chen, Fanglan and Du, Bowen and Lu, Chang-Tien},
booktitle={European Conference on Computer Vision},
pages={73--90},
year={2022},
organization={Springer}
}
Acknowledgement
The implementation is based on HSNet.
References
[1] Demir, I., Koperski, K., Lindenbaum, D., Pang, G., Huang, J., Basu, S., Hughes, F., Tuia, D., Raskar, R.: Deepglobe 2018: A challenge to parse the earth through satellite images. In: The IEEE Conference on Computer Vision and Pattern Recog- nition (CVPR) Workshops (June 2018)Li, X., Wei, T., Chen, Y.P., Tai, Y.W., Tang, C.K.: Fss-1000: A 1000-class dataset for few-shot segmentation. In: Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition. pp. 2869–2878 (2020)
[2] Codella, N., Rotemberg, V., Tschandl, P., Celebi, M.E., Dusza, S., Gutman, D., Helba, B., Kalloo, A., Liopyris, K., Marchetti, M., et al.: Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic). arXiv preprint arXiv:1902.03368 (2019)
[3] Tschandl, P., Rosendahl, C., Kittler, H.: The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific data 5, 180161 (2018)
[4] Candemir, S., Jaeger, S., Palaniappan, K., Musco, J.P., Singh, R.K., Xue, Z., Karargyris, A., Antani, S., Thoma, G., McDonald, C.J.: Lung segmentation in chest radiographs using anatomical atlases with nonrigid registration. IEEE trans- actions on medical imaging 33(2), 577–590 (2013)
[5] Jaeger, S., Karargyris, A., Candemir, S., Folio, L., Siegelman, J., Callaghan, F., Xue, Z., Palaniappan, K., Singh, R.K., Antani, S., et al.: Automatic tuberculosis screening using chest radiographs. IEEE transactions on medical imaging 33(2), 233–245 (2013)
[6] Li, X., Wei, T., Chen, Y.P., Tai, Y.W., Tang, C.K.: Fss-1000: A 1000-class dataset for few-shot segmentation. In: Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition. pp. 2869–2878 (2020)