genie-toolkit
genie-toolkit copied to clipboard
Bad values in databases result in awkward agent responses
For example, in this dialogue (obtained by running genie simulate-dialog
on dialogue MUL1167.json
of MultiWOZ), missing values for opening hour are directly copied to the agent response. In general, we should have some quality checks befor including values in agent responses. Values that are too long/short or don't satisfy a set of heuristics based on the column type should not be used in the agent response.
U: yeah um how about i can see camboats .
UT: $dialogue @org.thingpedia.dialogue.transaction . execute ; @uk.ac.cam.multiwoz.Attraction . Attraction ( ) filter type =~ " boat " ;
.
.
.
C: @uk.ac.cam.multiwoz.Attraction.Attraction() filter type =~ "boat"
C: #[results=[
C: { ... openhours="?", ... },
C: { ... openhours="?", ... },
C: ... ]];
A: I have found camboats, riverboat georgina and scudamores punting co. They're boats place to visit in cambridge which opens?
AT: $dialogue @org.thingpedia.dialogue.transaction.sys_recommend_three;
This is a MultiWOZ-specific issue. In reality, skill implementation must not return a value if it's not valid: they should return null or undefined, in which case the agent will not speak about that slot.