Unsupervised-Aspect-Extraction icon indicating copy to clipboard operation
Unsupervised-Aspect-Extraction copied to clipboard
verified publisher icon ruidan

coherence score

Open RaymondLi1101 opened this issue 6 years ago • 2 comments

Hi Ruidan, First of all, really great work. I tried it on some really noisy data set and some really cohesive aspects popped out. I do find the coherence score as a very interesting way to evaluate this unsupervised model. But I did not find the corresponding code in your evaluation(Or I missed it?). So I am wondering how did you compute the coherence score.

RaymondLi1101 avatar Jul 02 '19 18:07 RaymondLi1101

Hi, I did not include the coherence score computation in the code, you can find how it could be computed in Section 5.1.1 in the paper. However I do not think it is a good evaluation metric, as I found it does not well reflect the coherence in certain cases according to my personal experience.

ruidan avatar Jul 03 '19 08:07 ruidan

thanks for the reply. Would you mind share some of your experience with coherence score and how it can't reflect the coherence?

RaymondLi1101 avatar Jul 03 '19 13:07 RaymondLi1101