Sparse-Adversarial-Attack
Sparse-Adversarial-Attack copied to clipboard
wubaoyuan
G is updated to nan
I'm attacking the imageNet image. I used the Inceptionv3 pre-training model provided by Pytorch.
The following parameters are used for normalization:
mean = torch.Tensor([0.485, 0.456, 0.406]).view(1,3,1,1)
std = torch.Tensor([0.229, 0.224, 0.225]).view(1,3,1,1)
normalized_ops = (mean, std)
I set args.k to 5961.
When the update_G function is executed, all elements of G become nan.
What went wrong? Why does G become nan?
Hello, I've faced the same problem. After a few iterations, G is updated to 3.332902551362248e+44.
Maybe there are some small issues in the G's code.
In ADMM, the value of G is not constrained well, but after ADMM, G is hard-coded
G = (G > 0.5).float() #finetune
I don't know the exact reason. But a trick in the function update_G, performing G = (G>0.5).float() would ease the problem.
or you can modify the hyperparameter cur_rho.
I think I have addressed the issues. In the iteration process, if G is higher than 1, the ADMM algorithm will result in exploding gradient problem. So just add torch.clamp(G, 0, 1) after each iteration.
I think I have addressed the issues. In the iteration process, if G is higher than 1, the ADMM algorithm will result in exploding gradient problem. So just add torch.clamp(G, 0, 1) after each iteration.
Thank you for your help! However, I have carried out torch.clamp(G, 0, 1) after each iteration as you said, but G will still be updated to nan. Could you please provide more detailed code? Thank you very much！
Gradient explosion happens when the first step of update_G or after some steps of update_G? Maybe you can also try to add .sign() to the G.grad.data and force the NAN value to be 0. o(╥﹏╥)o Actually, when I try these methods, some examples get worked. But, bad examples still exist.
Hi! Do you know how to tune the parameters for imagenet so that it can achieve the statistics reported in their paper? I currently have much larger l1, l2 and l_inf norm than the statistics in their paper when I choose a similar k.
I think I have addressed the issues. In the iteration process, if G is higher than 1, the ADMM algorithm will result in exploding gradient problem. So just add torch.clamp(G, 0, 1) after each iteration.
Thank you for your help! However, I have carried out torch.clamp(G, 0, 1) after each iteration as you said, but G will still be updated to nan. Could you please provide more detailed code? Thank you very much！
Hi! Recently, I tune the hyperparameters many times. When I increase the ratio of the grad of Loss/G, G successfully reaches the target sparsity. So I think finetuning the ratio of the grad of Loss/G is the solution to avoid NAN problems.
Hello, I am always unable to run ImageNet with the code provided by the author. Could you please tell me some relevant parameter settings when you run the ImageNet dataset?
