Adversarial Generative Network (AGN) implementation in PyTorch

Please see the paper by Sharif et al. for more information.

Also check out my accompanying blog post.

Testing your adversarial generator with your webcam

File video_inference_cv.py presents a method to test out your adversarial generator (alongside a simple facial recognition classifier) using your webcam.

$ python video_inference_cv.py --file camera

Press the p key on your keyboard to swap generators and create a new set of glasses.

Overview of Methods

Goal

to generate eyeglass frames that make a facial recognition classifier think you are someone else

Tools

• 1 GAN
• 1 classifier
• 1 big dataset of eyeglass frames
• some images of celebrity faces
• some images of my face

Algorithm

Please see paper for more info, but here is my brief overview of the Algorithm 1 from the paper:

Finetuning the Facial Recognition Network

Pre-train GAN

Prepare Data for AGN Training

AGN Training

For video pre-processing steps

brew install imagemagick

for szFile in ./Michael_Chaykowsky/*.png
do 
    magick mogrify -rotate 90 ./Michael_Chaykowsky/"$(basename "$szFile")" ; 
done

pip install autocrop

$ autocrop -i ./me_orig/Michael_Chaykowsky -o ./me/Michael_Chaykowsky160 -w 720 -H 720 --facePercent 80

Directory must have subdirectory with name of person and images stored inside

• this is already done by the previous autocrop step

! pip install tensorflow==1.13.0rc1

! pip install scipy==1.1.0

Use the directory above the classes directory (not the actual directory holding the images)

for N in {1..4}; do \
python ~/Adversarial/data/align/align_dataset_mtcnn.py \ # tensorflow script
~/Adversarial/data/me/ \ # current directory
~/Adversarial/data/me160/ \ # new directory
--image_size 160 \
--margin 32 \
--random_order \
--gpu_memory_fraction 0.25 \
& done

with open('bboxes_fnames_test.csv', 'w') as f:
    writer = csv.writer(f)
    writer.writerows(zip(['filename']))
    a = [os.path.basename(x) for x in glob.glob('agn_me_extras160/Michael_Chaykowsky/*.png')]
    writer.writerows(zip(a))

Directory structure

project
│   README.md
│   AGN.ipynb  
│
└───data
│   │   files_sample.csv
│   └───eyeglasses
│   │
│   └───test_me
│       └───train
|           └───Adrien_Brody
|           ...
|           └───Michael_Chaykowsky
|           ...
|           └───Venus_Williams
│       └───val
|           └───Adrien_Brody
|           ...
|           └───Michael_Chaykowsky
|           ...
|           └───Venus_Williams
│   
└───models
│   │   inception_resnet_v1.py
│   │   mtcnn.py
│   └───utils

Run example

$ python main.py --last_layers n --ngpu 0 --testing y --write_vid n

Refrences

@article{Sharif_2019, title={A General Framework for Adversarial Examples with Objectives}, volume={22}, ISSN={2471-2574}, url={http://dx.doi.org/10.1145/3317611}, DOI={10.1145/3317611}, number={3}, journal={ACM Transactions on Privacy and Security}, publisher={Association for Computing Machinery (ACM)}, author={Sharif, Mahmood and Bhagavatula, Sruti and Bauer, Lujo and Reiter, Michael K.}, year={2019}, month={Jul}, pages={1–30} }

@misc{cao2018vggface2, title={VGGFace2: A dataset for recognising faces across pose and age}, author={Qiong Cao and Li Shen and Weidi Xie and Omkar M. Parkhi and Andrew Zisserman}, year={2018}, eprint={1710.08092}, archivePrefix={arXiv}, primaryClass={cs.CV} }