MultiViz icon indicating copy to clipboard operation
MultiViz copied to clipboard

About the Model simulation on MOSEI dataset.

Open Columbine21 opened this issue 2 years ago • 0 comments

Thanks for your great work of visualizing and understanding internal mechanics in multimodal models. However, there is still some concerns with regard to affective analysis topic.

  1. Model simulation shows that 100% Correctness and 100% Agreement on MOSEI dataset, is there some detailed example (entire evidence for affective judgement) of such cases? Does that means the interpretability of MSA models is trivial? In fact, when i interact with the provided online website, i can not really judge the sentiment well.
  2. In Appendix D.3 (examples of human-assigned concepts), the subsection of affective computing concepts are missing. What human-assigned non-verbal concepts are verified as important for sentiment analysis (only Language concept: positive mentions in text are shown in Figure 4.)
  3. For error analysis of sentiment analysis, how to separate the Cross-modal interaction errors and prediction error, as there is no perceptible of Cross-modal interaction in affective computing.

Thanks for your great works again, I am looking forward to your reply.

Columbine21 avatar Jun 01 '23 02:06 Columbine21