natural-adv-examples

natural-adv-examples copied to clipboard

Mismatch between the ImageNet-A classes reported in the paper and in the download?

1

Hi, thanks for providing this dataset! I downloaded ImageNet-A following the link in the README of this repository and had a look at the provided README to see which classes...

expectopatronum

Hi. Hendrycks. Can you provide instructions to actually test custom pretrained models?

wjun0830

Hello, I am trying to run eval_many_models.py, I commented out all but resnet50 to get the publication comparison, when I loop over: _for net_params in models_to_test:_ I get: ('pytorch/vision', 'resnet50')...

gjkunde

Fixing a severe MISTAKE in calculating the accuracy using ImageNet-A

2

The eval.py example provided along with the release of ImageNet-A dataset has a severe calculation mistake in calculating the ImageNet-A accuracy given a ImageNet-1K pretrained model. Cause: ImageNet-1K has 1,000...

HengyueL

Thank you for the dataset. What are the values in a file name? For example, in this one, `0.000116_digital clock _ digital clock_0.865662`, the last value seems to be the...

fugokidi

Hello, thanks for the interesting work. Which resnet50 models did you use? Specifically, was the ResNet50_Weights.IMAGENET1K_V2 of pytorch https://pytorch.org/vision/main/models/generated/torchvision.models.resnet50.html used or can it still be evaluated as an unknown resnet50?...

ThomasNorr

The use of `range(len(bins)-1)` actually missed out calculating the last bin `74`, hence the `RMS Calib Error` is incorrect

zhulinchng