ConvLab-2 icon indicating copy to clipboard operation
ConvLab-2 copied to clipboard

[Feature] end-to-end test with CrossWOZ

Open nuance1979 opened this issue 4 years ago • 1 comments

Describe the feature I was able to run python tests/test_BERTNLU-RuleDST-RulePolicy-TemplateNLG.py and got the same results as in README. Thanks a lot!

However, as soon as I tried to run the same test for CrossWOZ, I ran into the issue that there was no evaluator for CrossWOZ dataset. In fact, currently analyzer only works with the default MultiWOZ evaluator.

I tried to use MultiWOZ evaluator but it didn't work. I looked at CrossWOZ repo but it seemed there wasn't an end-to-end evaluator class, either.

Expected behavior A working end-to-end evaluator for CrossWOZ dataset.

Additional context n/a

nuance1979 avatar May 12 '20 23:05 nuance1979

Yes, we will add it in the future

zqwerty avatar May 13 '20 01:05 zqwerty