emoca icon indicating copy to clipboard operation
emoca copied to clipboard

Published 20 hours ago • verified publisher icon radekd91

Not good for wink expression

Open wwdok opened this issue 2 years ago • 3 comments

I test EMOCA with some wink expression images, but results seems not good : wink_girl inputs geometry_detail

wink_ironman geometry_detail inputs

wink_obama geometry_detail inputs

Do you have some ideas to improve the result ?

wwdok avatar Jul 07 '22 12:07 wwdok

Sadly, this is true. As it so happens, winks like these are not very well represented in the AffectNet dataset as they don't necessarily correspond into one of the basic expressions (happy, sad, angry, ...). As a consequence, the emotion network is not trained to recognize these and therefore it does not supervise EMOCA to reconstruct it well.

A better, more finegrained emotion dataset would help alleviate this limitation. The emotion network would have to be retrained as well as EMOCA. I'm afraid there is no easy way to hack this in code-wise. It's more of a data issue.

radekd91 avatar Jul 08 '22 12:07 radekd91

A small update, EMOCA itself and the AffectNet dataset may actually have some wink-like expressions. I have been informed, that inability to create wink shapes might run deeper than that - it seems that the FLAME face model's training data did not really contain any winks and therefore it would not be able to generate these very well.

radekd91 avatar Jul 27 '22 10:07 radekd91

Update, @wwdok , I recommend giving the newly release EMOCA 2.0 a try. It will most likely do a lot better. Thanks again for your feedback

radekd91 avatar Feb 19 '23 12:02 radekd91