multiwoz icon indicating copy to clipboard operation
multiwoz copied to clipboard

Published 20 hours ago • verified publisher icon budzianowski

Evaluation Dataset in "Response Generation" - "end-to-end models" part

Open nqchieutb01 opened this issue 1 year ago • 1 comments

Is this part used multiwoz 2.2 or multiwoz 2.0 as a benchmark dataset. I'm so confused, in RewardNet, Mars, KRLS original paper, all results are the same as your table, but they all reported in multiwoz 2.0 dataset. Morever, in the TOATOD paper, authors reported combined score in multiwoz 2.2 dataset. Is there any mistakes. Can you explain this inconsistent. Thanks !

nqchieutb01 avatar Sep 20 '23 03:09 nqchieutb01

Is this part used multiwoz 2.2 or multiwoz 2.0 as a benchmark dataset. I'm so confused, in RewardNet, Mars, KRLS original paper, all results are the same as your table, but they all reported in multiwoz 2.0 dataset. Morever, in the TOATOD paper, authors reported combined score in multiwoz 2.2 dataset. Is there any mistakes. Can you explain this inconsistent. Thanks !

I have the same doubt.

comprehensiveMap avatar Nov 03 '23 16:11 comprehensiveMap