Cannot achieve good restoration effect on RGB images

[mankasto]

Hi, thank you very much for your work. I am more interested in gradient inversion attacks, especially the work from the paper "See through Gradients: Image Batch Recovery via GradInversion". Referring to your documentation, I got good results with a smaller batch size on the MNIST dataset, but switched to RGB image datasets like cifar100, the effect was very unsatisfactory. I would like to ask if you have successfully reproduced the effect in the paper or have any suggestions for the setting of hyperparameters? I'll be grateful if you can reply as soon as possible.

[Koukyosyumei]

@mankasto Thank you for your interest in our project! Could you share your current model architecture, batch size, and other hyperparameters? I also would like to know what your reconstructed images looks like.

Based on my experiences, cossim loss might works more stable than l2 distance when reconstructing RGB images, but it's just my heuristic.

[mankasto]

Thank you for your prompt reply. The experimental setup I used is as follows:

1. The model architecture adopts the simple LeNet network from the examples in your project.
2. batch_size=3, distance_function=L2, tv_reg_coef=1e-6, optimizer=Adam, learning_rate=1e-3, num_iterations=2000, and the rest kept default settings.
3. Original image and restored result：

[Koukyosyumei]

@mankasto Sorry for the late response. I have prepared a notebook that shows the example of the reconstruction of CIFAR-100 with multiple batch sizes. I didn't tune hyper-parameters, so you might be able to improve them.

https://colab.research.google.com/drive/1KShVaSQd_4uqZk9l2OQQ-imszfimawf9?usp=sharing

The original paper "See through Gradients: Image Batch Recovery via GradInversion" uses the scale parameter $\alpha_G$, which AIJack does not support. So, you may need to re-scale other parameters like TV weight.

I guess supporting $\alpha_G$ is easy, so I would like you to create a pull request for it if you are interested.

Example (batch_size = 5)

true

reconstructed images

[Koukyosyumei]

@mankasto

Have you solved this issue?