Feature Space Singularity for Out-of-Distribution Detection

This repository is the implementation of the paper Feature Space Singularity for Out-of-Distribution Detection.

FSSD algorithm

The central idea of FSSD algorithm is to use the following FSSD score to tell OoD samples from in-distribution ones:

where is the feature extractor and is the Feature Space Singularity (FSS). We approximate FSS simply by calculting the mean of uniform noise features: .

Implemented Algorithms

In this repository, we implement the following algorithms.

Note that OE shares the same implementation with Baseline. To test these methods, see the Evaluation section. We also welcome contributions to this repository for more SOTA OoD detection methods.

Requirements

Python: 3.6+

To install required packages:

pip install -r requirements.txt

Datasets

You can download FashionMNIST and CIFAR10 datasets directly from links offered in torchvision. For ImageNet-dogs-non-dogs dataset, please download the dataset from this link and find the dataset description in this issue. (We will add more datasets and corresponding pre-trained models in the future.)

The default dataset root is /data/datasets/.

Training

To train the model(s) in the paper, see lib/training/. We provide the training codes of FashionMNIST and CIFAR10 datasets.

Evaluation

To evaluate OoD detection performance of FSSD, run:

python test_fss.py --ind fmnist --ood mnist --model_arch lenet --inp_process

You can also evaluate OoD performance of other methods:

Baseline:

python test_baseline.py --ind fmnist --ood mnist --model_arch lenet

ODIN:

python test_odin.py --ind fmnist --ood mnist --model_arch lenet

Mahanalobis distance:

python test_maha.py --ind fmnist --ood mnist --model_arch lenet

Deep Ensemble:

python test_de.py --ind fmnist --ood mnist --model_arch lenet

Outlier Exposure:

python test_baseline.py --ind fmnist --ood mnist --model_arch lenet --test_oe

Pre-trained Models

You can download pretrained models here:

• Google cloud links for models trained on FashionMNIST, CIFAR10 , ImageNet-dogs-non-dogs-dataset,using parameters specified in our supplements. More pre-trained models will be added in the future.

Please download the pre-trained models and put them in pre_trained directory

Results

Our model achieves the following performance on :

OoD detection benchmarks

FashionMNIST vs. MNIST:

CIFAR10 vs. SVHN:

Citation

If you find this repository helpful, please cite:

@inproceedings{huang2021feature,
  title={Feature Space Singularity for Out-of-Distribution Detection},
  author={Huang, Haiwen and Li, Zhihan and Wang, Lulu and Chen, Sishuo and Dong, Bin and Zhou, Xinyu},
  booktitle={Proceedings of the Workshop on Artificial Intelligence Safety 2021 (SafeAI 2021)},
  year={2021}
}

Contributing

The usage and contribution to this repository is under MIT licence.