FlowNetC Normalization & Weights

[SimonSchrodi]

Hi,

I have a couple of questions regarding flowattacks on FlowNetC:

• Is there a reason why you normalize the inputs for FlowNetC to [-1,1]? I am wondering why this is, because in NVIDIA's repository they normalize the inputs differently using the rgb mean and rgb max.
• In your experiments in the paper, do you also use the pretrained weights or did you retrain FlowNetC?

[anuragranj]

We found that it barely affected results, it was within 1% or so. When you subtract the mean and divide by max, assuming all images have the same range [0,1], or [0,255]; it would result in the same image tensor within [-1, 1] after normalization.

Also, it is very much possible that the numbers would be different. We noticed a really large variance in the numbers with different seeds. So we ran the training for the patch several times and picked the best attack for each.

Now, if your objective is to get higher numbers for attack, you could just remove augmentations. But then, when you remove augmentations, the attack doesn't generalize to real world. There is a trade-off and you can vary the amount of augmentation, or you can completely remove the augmentation by using the flag --norotate, to get higher numbers. We saw much higher numbers with no augmentations; but haven't used them in the paper.

Also, we used pretrained weights.

[SimonSchrodi]

Thanks for the answer.

So I noticed this too. However, difference is a bit larger for me, i.e. 11.5 unattacked EPE (using mean/max) and 14.52 unattacked EPE (normalize to [-1,1]) on KITTI 2015 train (without attack). Note that using mean/max is closer to the reported EPE (11.36) in the FlowNet2 paper.

How many runs did you try to obtain the "best" adversarial patch?

How can this be the same using mean & max? If I am not mistaken mean and max then would be both 0.5. However, this would not make too much sense, since basically the image then would be just a gray scale image. I think it is more in the range of [-0.4, 0.6] (this also corresponds to FlowNet2C's internal data normalization).

[anuragranj]

Okay, i think that's a valid issue. We did get around 14 EPE on unattacked FlownetC yes. We did at least 5-10 runs for each.