KB-NER icon indicating copy to clipboard operation
KB-NER copied to clipboard

Which table result is the real result

Open JinFish opened this issue 2 years ago • 4 comments

According to the description of the paper, ITA-ALL+CVA in Table 2 should be the performance of the model, but in Table 4 the performance of ITA-ALL+CVA is different from that in Table 2. May I ask which table's result is the true result and why the two tables' results are different?

JinFish avatar Sep 20 '22 09:09 JinFish

Hi, the result in Table 1 is the averaged result over 5 times. In table 2, since most of these previous approaches report the best model accuracy instead of the averaged model accuracy, we use the best model accuracy of ITA-All+CVA over 5 runs. You may read section 3.2 for more details.

wangxinyu0922 avatar Sep 20 '22 10:09 wangxinyu0922

Thank you for your reply, but I would like to ask why the results of table 2 and table 4 are not consistent and the results of table 4 and table 1 are not consistent. What you replied to me is the reason why the results of table 1 and table 2 do not agree.

JinFish avatar Sep 20 '22 11:09 JinFish

Oh, in table 4, I put our initial results of Twitter 15/17. The models are trained with LA+GA+CVA. I will fix this issue in our newest arxiv version. You may take the results in Table 1 and Table 2.

wangxinyu0922 avatar Sep 20 '22 11:09 wangxinyu0922

Thank you very much!

JinFish avatar Sep 20 '22 11:09 JinFish