CMU-MOSI: Manual gesture annotations

[ATriantafyllopoulos]

In Multimodal sentiment intensity analysis in videos: Facial gestures and verbal messages, there's reference to manual gesture annotations: smile, frown, head shake, head nod.

Are these available through the SDK, or at least as raw metadata?

[A2Zadeh]

Hi @ATriantafyllopoulos,

These are buried somewhere :). We immediately moved to automated annotations, but if you are really interested, I will try to find them.

[ATriantafyllopoulos]

Well, we are interested in smile in particular, as we expect it to have an impact on acoustic features. As such:

a) It's very interesting to detect it from acoustic features alone b) It's interesting to see how it interacts with acoustic features and sentiment

Since I am not aware of many datasets that have this information available as ground truth, I think it would make for a very interesting use-case. I'd be much obliged if you can dig it up!