detectors_review
detectors_review copied to clipboard
The results of LID for detecting PGD-L∞(8) is much lower than the paper
HI!
I want to get the results of each type of adversarial example detection as the baseline. Because the max_iter should be 5 in my paper rather than the default number 100, I run the code to generate the results.
However, when I ran the code (only comment out the code for other attacks, the max_iter is still 100), the results of LID for detecting PGD-L∞(8) was much lower than the paper (the followingscreen shot).
The results should be DR=94.39 and FPR=1.81, but I got DR=66.13 and FPR=11.29. Is the setting of the paper is different from the code? Or do you know anything may be wrong? the code I fork and change is here: https://github.com/CycleBooster/detectors_review
By the way, I change common/utils.py and generate_adv.py. Otherwise, the newest version of art can't run.
I train the model and LID again and get a much different result.
Although the result still lower than the paper, it becomes much better! Does that mean the accuracy of LID is very unstable?