BotFramework-Composer icon indicating copy to clipboard operation
BotFramework-Composer copied to clipboard

Dialog's "Intent recognized" trigger does not kick in when a winning intent with low score determined by Orchestrator

Open dawwa opened this issue 3 years ago • 2 comments

Describe the bug

Dialog cannot be launched when it has same utterances as Orchestrator for an intent, but this intent is determined with low score, even it is a winning intent. e.g., When 3 intents X, Y, Z with scores are "X=0.4, Y=0.2, Z=0.1", the winning intent is X, even it's lower than 0.5

Setup a DialogForIntentX with "Intent recognized" trigger, and have the same utterances as in Orchestrator for intent X. then Begin a new dialog for DialogForIntentX in Orchestrator's Intent recognized trigger, e.g., OnIntent(X) Talk to the bot with something that can cause low score and Orchestrator has to pick a winner. See example in the screen shot.

P.S., If I could think of a mitigation for this problem, that would be Is there a way to set a threshold to make Orchestrator trigger the Unknown intent trigger when all candidate intents are with low scores? I spot this problem when I was expecting my test message would fall into the Unknown intent while it's actually being directed to a different dialog but did not response due to the winning intent's low score and the reasons illustrated below.

This feels complex to explain in text, but I tried to illustrate in the screenshot below.

Version

2.1.2

OS

  • [ ] macOS
  • [X] Windows
  • [ ] Ubuntu

To Reproduce

Steps to reproduce the behavior:

  1. Create a bot with Orchestrator at root dialog, with sample utterances as shown below (without the - how are you line)
  2. Add another dialog, with LUIS Intent recognized trigger, with sample utterances as shown below (without the - how are you line)
  3. Talk with the bot with utterance how are you
  4. Add - how are you in the LUIS Intent recognized trigger, but not in the Orchestrator sample utterances, and try talk to the bot again with utterance how are you

Actual behavior

The dialog can only be triggered for Setup # 4

Expected behavior

The dialog can be triggered for both Setup # 3 and Setup # 4

Screenshots

image

Additional context

dawwa avatar Feb 10 '22 09:02 dawwa

@dawwa This is intended behavior. To solve your issue, you have to set the condition for each intent like this : image

This will make it so that your intent will not trigger unless the recognition threshold is above 0.8.

johnkm516 avatar Feb 10 '22 10:02 johnkm516

thanks @johnkm516 , this seems reasonable, and it looks a promising solution!

dawwa avatar Feb 10 '22 19:02 dawwa