genie-toolkit icon indicating copy to clipboard operation
genie-toolkit copied to clipboard

Bad values in databases result in awkward agent responses

Open s-jse opened this issue 3 years ago • 1 comments

For example, in this dialogue (obtained by running genie simulate-dialog on dialogue MUL1167.json of MultiWOZ), missing values for opening hour are directly copied to the agent response. In general, we should have some quality checks befor including values in agent responses. Values that are too long/short or don't satisfy a set of heuristics based on the column type should not be used in the agent response.

U: yeah um how about i can see camboats .
UT: $dialogue @org.thingpedia.dialogue.transaction . execute ; @uk.ac.cam.multiwoz.Attraction . Attraction ( ) filter type =~ " boat " ;
.
.
.
C: @uk.ac.cam.multiwoz.Attraction.Attraction() filter type =~ "boat"
C: #[results=[
C:   { ... openhours="?", ... },
C:   { ... openhours="?", ... },
C:   ... ]];
A: I have found camboats, riverboat georgina and scudamores punting co. They're boats place to visit in cambridge which opens?
AT: $dialogue @org.thingpedia.dialogue.transaction.sys_recommend_three;

s-jse avatar Feb 04 '21 02:02 s-jse

This is a MultiWOZ-specific issue. In reality, skill implementation must not return a value if it's not valid: they should return null or undefined, in which case the agent will not speak about that slot.

gcampax avatar Feb 04 '21 02:02 gcampax