ConvLab-2
ConvLab-2 copied to clipboard
[Feature] end-to-end test with CrossWOZ
Describe the feature
I was able to run python tests/test_BERTNLU-RuleDST-RulePolicy-TemplateNLG.py
and got the same results as in README. Thanks a lot!
However, as soon as I tried to run the same test for CrossWOZ, I ran into the issue that there was no evaluator for CrossWOZ dataset. In fact, currently analyzer only works with the default MultiWOZ evaluator.
I tried to use MultiWOZ evaluator but it didn't work. I looked at CrossWOZ repo but it seemed there wasn't an end-to-end evaluator class, either.
Expected behavior A working end-to-end evaluator for CrossWOZ dataset.
Additional context n/a
Yes, we will add it in the future