cvpr18-caption-eval icon indicating copy to clipboard operation
cvpr18-caption-eval copied to clipboard

Published 20 hours ago • verified publisher icon richardaecn

Inter-human score

Open Yudezhi opened this issue 4 years ago • 2 comments

I can not find the code to calculate the Inter-human score. I use "kendalltau" provided by "scipy.stats" to get the mean value(0.739) of AB and AC which is not matched with yours. (ABC is assumed as three scorers in Flickr8k) However, if I keep 2 decimal places, the result is 0.73 which matched with the metric "SPICE". Wish for your reply~

Yudezhi avatar Dec 16 '20 09:12 Yudezhi

I can not find the code to calculate the Inter-human score. I use "kendalltau" provided by "scipy.stats" to get the mean value(0.739) of AB and AC which is not matched with yours. (ABC is assumed as three scorers in Flickr8k) However, if I keep 2 decimal places, the result is 0.73 which matched with the metric "SPICE". Wish for your reply~

I have the same question. Have you solved this problem?

qiantianwen avatar Sep 29 '21 14:09 qiantianwen

I can not find the code to calculate the Inter-human score. I use "kendalltau" provided by "scipy.stats" to get the mean value(0.739) of AB and AC which is not matched with yours. (ABC is assumed as three scorers in Flickr8k) However, if I keep 2 decimal places, the result is 0.73 which matched with the metric "SPICE". Wish for your reply~

I have the same question. Have you solved this problem?

嗯,最后解决了. 论文中给出的结果是正确的,印象中好像是计算三个人之间的,也就是计算三次.时间太长了,具体代码忘记了.

Yudezhi avatar Sep 30 '21 02:09 Yudezhi