Chiralization/mirroring (new functionality)

[Allexxann]

I'm asking you to add a variant analysis that can measure Aus of face sides independently. Re-training the whole network would be a hassle, but there's an easier method. The rough algorithm is as follows: take a detected face, mirror it across a middle line, taking the pose into account, then extract AUs for the original and both reflections separately.

Here's the general idea - the face aligned in Openface and mirrored images on both sides.

It can also be seen that here the alignment is rather poor, but mirroring was done manually, without using facial landmarks.

[ljchang]

Thanks for the suggestion @Allexxann. We will look into this for the next release. Just as a heads up we are also testing alternative models for landmarks and AUs in a future release that will be able to detect lateralized expressions.

[Allexxann]

I don't know Python, but one of our students finally wrote it. code.zip Here are program files (there's a stupid bug included - my mistake, I was trying to skip already analyzed files).

Well, good news - it kinda workds. Bad news - in about 12 hours it analyzed about 10 000 frames from one video. The video in question is 44 000 frames long, and I have about 200 of them to analyze in only the most urgent batch. The bottleneck is apparenty the CPU, not memory, and I have 11th Gen Intel(R) Core(TM) i7-11800H. I guess it doesn't actually work, but it was a nice try.

Currently we're trying to mess with CUDA, but I'd much appreciate a proper neural net, so that there would be no need to cast the whole detection pipeline 4 times on the same frame.