UBAR-MultiWOZ
UBAR-MultiWOZ copied to clipboard
TonyNemo
Multiwoz evaluation
Hi, thank you for sharing the code! I appreciate your work and I have a question.
The original MultiWoz evaluate.py calculates the INFORM score by checking if the first offered venue is correct or not. elif len(venue_offered[domain]) > 0 and venue_offered[domain][0] in goal_venues: match += 1 However your evaluation code is checking if there's an overlap between offered venues and goal venues. elif len(venue_offered[domain]) > 0 and len(set(venue_offered[domain])& set(goal_venues))>0: match += 1 Doesn't it lead to a higher score than the score obtained by the original evaluation code?
