How to add gaussian nosie images and uniform noise images to the test set to compute the AUROC against CIFAR10 when using CIFAR10 as in-distribution data?

[Emilia-Re]

Thank you for your great work and efforts for building this project.
When I run the code ,I want to know how to add additional ood data(like noise images) to the test set to compute the AUROC against the ID data. Besides, I didn't find the document of this project ,and it would be much better for me to understand and use this package if I have one. Under the ./benchmark_imglist/cifar10 directory,there are many .txt files, what are these .txt files used for? It would be highly appreciated if you could offer an answer.

[zjysteven]

Hi, those .txt files store and help index all relevant images. For example for CIFAR-10 (ID) v.s. CIFAR-100 (OOD), the corresponding txt files benchmark_imglist/cifar10/test_cifar10.txt and benchmark_imglist/cifar10/test_cifar100.txt store the relevant ID and OOD test images.

To test on a new dataset that is not currently encoded in OpenOOD, we encourage you to create a txt file similarly, and encode the dataset information in the DATA_INFO dictionary in openood/evaluation_api/datasets.py (see https://github.com/Jingkang50/OpenOOD/blob/be315e6e37e48bf0451576c0c43dcda06dcd8d08/openood/evaluation_api/datasets.py#L18-L84 for the example of CIFAR-10 as ID). After that, you should be able to evaluate on custom OOD datasets by using our new evaluator.