DeepWarp for Facial Expression Manipulation

Warning: the repo is under construction so the code may not be runnable out of the box!

This is a repository containing torch7 code implementing DeepWarp. Be sure to check out the online demo. Due to licensing restrictions I'm not able to release the original gaze manipulation code and corresponding dataset, but the model presented here is very similar and you are free to use it as a reference point.

Showcase

To showcase general applicability of the approach, I'm using an ever so slightly modified network to handle smile addition and removal in the images containg human faces.

Model

The model (shown above) contains three networks:

• Transformer T. Warps the image given δ which is either +1 (add smile) or -1 (remove smile)
• Discriminator D. Makes sure that the warped output looks plausible.
• Attribute branch A. Infers attribute (1 for "smile", -1 for "no smile") value given an image.

The training objective is a weighted sum of two components:

where the first term is called the attribute loss and is a standard GAN cost.

Citation

Please cite the DeepWarp paper in your publications if this repo helps your research:

@inproceedings{ganin2016deepwarp,
  title={DeepWarp: Photorealistic image resynthesis for gaze manipulation},
  author={Ganin, Yaroslav and Kononenko, Daniil and Sungatullina, Diana and Lempitsky, Victor},
  booktitle={European Conference on Computer Vision},
  pages={311--326},
  year={2016},
  organization={Springer}
}