NLU.DevOps icon indicating copy to clipboard operation
NLU.DevOps copied to clipboard

Determine how best to handle entity confusion results given speech mis-recognitions.

Open rozele opened this issue 6 years ago • 1 comments

If I have a test case like:

{
  "text": "contact info for carole",
  "intent": "ContactInfo",
  "entities": [
    {
      "matchText": "carole",
      "entityType": "builtin.personName"
    }
  ]
}

And through speech transcription, we instead get something like:

{
  "text": "contact info for carol",
  "intent": "ContactInfo",
  "entities": [
    {
      "matchText": "carol",
      "entityType": "builtin.personName"
    }
  ]
}

Currently, this is reported as a missing entity.

rozele avatar Dec 21 '19 01:12 rozele

This issue is probably not needed. We can't effectively test ASR -> entity results unless we are testing an end-to-end system and the entities are fully disambiguated.

The correct approach here is to label the ASR output instead of the expected output.

I will file another issue to consider whether we want to remove speech -> LUIS result testing.

rozele avatar Feb 03 '20 19:02 rozele