FaceSwap icon indicating copy to clipboard operation
FaceSwap copied to clipboard

Published 20 hours ago • verified publisher icon MarekKowalski

DAN, 3d Model and eyes question

Open oleg-mir opened this issue 6 years ago • 3 comments

Hi, Great job and thank you for sharing! I have some follow-up questions after working with your project:

  1. I was trying the DAN version, but i was unable to make it work with GPU. Is it should be enough to tell to theano use GPU via THEANO_FLAGS='device=cuda,floatX=float32'? http://deeplearning.net/software/theano/tutorial/using_gpu.html

  2. In order to get the best results i want to edit the 3d mesh model. should the model match the source face or the targets?

  3. i wan't to implement eyes blinking. Could you advice what is the best approach for that?

Thanks in advance!

oleg-mir avatar Apr 23 '18 05:04 oleg-mir

Hi,

Here are my comments to your questions:

  1. My settings are set in the .theanorc file in the following way:
[global]
device=gpu
floatX=float32

  1. Ideally you would need to edit both, hard to say which one is more important. In general the edition of those meshes is quite difficullt to get right.

  2. I can think of two easy ways: a) Add a blendshape that closes the eyes. The issue of this method would be that in the source face there might be no texture corresponding to the eyelid region, because of that this method is prone to errors. b) Show the users real eyes instead of the eyes of the source face, just as is done with the mouth interior. To achieve this you would need to remove the parts of the mesh (the triangles) that correspond to the eye regions.

Marek

MarekKowalski avatar Apr 23 '18 19:04 MarekKowalski

thanks for the reply. I've managed to run on GPU using the .theanorc as you suggested.

regarding the eyes:

  1. I've did what you've suggested and removed the mesh that correspond to the eye regions. But how do I get from the .obj to .npy ?

  2. which version of the candide did you use? i see in the 3.1.6 version eyes closed implementation: "# AUV6 Eyes closed (AU42/43/44/45)"

  3. I've checked all .npy files but i could find those numbers at the 3.1.6 version? I'm i missing something or did you use a different version? http://www.icg.isy.liu.se/candide/candide3.wfm

oleg-mir avatar Apr 23 '18 20:04 oleg-mir

Hi,

  1. You need to know the indices of the triangles you removed from the obj mesh and remove the corresponding rows of the mesh variable in zad2.py
  2. Here is a piece of code that converts the candide model file into the npz. It is a very messy piece of code, just saying ;)
  3. I don't really understand this question.

Marek

MarekKowalski avatar Apr 25 '18 20:04 MarekKowalski