Did you do any experiments on WOZ2.0 dataset on STAR model?

Open qingyue2014 opened this issue 3 years ago • 2 comments

Did you do any experiments on WOZ2.0 dataset on STAR model? The STAR model is a strong baseline on MultiWOZ dataset. But I evaluated it on WOZ dataset(a single domain dataset) and found that the STAR can only achieve 87% JGA on WOZ dataset. It is a really bad performance, compared with SUMBT model (91.0%). I use the open codes which is provided from your GitHub. I couldn’t fight out the reason. Can you give me some advise or analysis?

Mar 21 '22 01:03 qingyue2014

Hi,

Thanks for your interest! I didn't test the method on WOZ2.0. I guess you might need to choose appropriate hyperparameters to get good results on this small dataset. The success of the STAR model also depends on the degree of correlations among slots. If the slot values don't have correlations, the model may not contribute to better performance.

Mar 21 '22 10:03 smartyfh

I got it， thx！

Mar 21 '22 10:03 qingyue2014