RobustVLM icon indicating copy to clipboard operation
RobustVLM copied to clipboard

CLIP image generation

Open realfolkcode opened this issue 1 year ago • 3 comments

Hi! Thank you for your work and congratulations on ICML acceptance! I am not sure if the issues is appropriate place but I wanted to share some of my findings regarding image generation with guidance from the gradients of Robust CLIP. Here I ran some experiments, and the results seem to be decent, which indicates that the gradients are perceptually-aligned. I thought you might find this interesting.

realfolkcode avatar Jun 11 '24 20:06 realfolkcode

Hi, Thanks for sharing. The generated images look very nice! We have also looked a bit into the interpretability of adversarial perturbations for robust CLIP as part of another project, which should be on arxiv soon :)

chs20 avatar Jun 12 '24 11:06 chs20

Great! Looking forward to that paper!

realfolkcode avatar Jun 12 '24 16:06 realfolkcode

You can refer to CLIPAG https://arxiv.org/abs/2306.16805. The paper has demonstrated this phenomenon. It might be helpful. @realfolkcode

dongdongunique avatar Jan 03 '25 06:01 dongdongunique