some questions about dropout in VAT

[caoxu915683474]

hello, I have some questions about dropout in VAT.

If I use dropout in VAT, the output distribution will change even without perturbation.

thanks!

[enricivi]

Hello Cao_enjun,

Excuse me, I can't understand the problem. Can you please provide a more detailed explanation? Is the same problem present in Adversarial Training (AT)?

Thank you, @enry12

[caoxu915683474]

Hi, I come back again. A question of virtual adversarial training arose to me. When using this method, we first feed the unlabeled data into the forward net to get their predictions as labels, and in the meantime, feed the noised data to get logits, then regard the KL divergence as the loss to get r_vadv. But in your codes, you applied dropout to the forward net. As far as I know, tensorflow only reuses variables and no operation (e.g. dropout) in the graph, thus the logits and the labels we get from the above procedure use different settings of dropout, i.e., the KL divergence computed may include the noises from dropout. How do you consider about it? Would it make the r_vadv not accurate enough?

[aurelpjetri]

Hi Cao_enjun, sorry for the late reply. We have analysed our code and your issue seems to be actually a bug in our code. Unfortunately, we can't currently fix this problem due to other deadlines. If you have a solution to propose and want to make a pull request we would appreciate it very much.

Thank you, aurel.