NLU.DevOps
NLU.DevOps copied to clipboard
Determine how best to handle entity confusion results given speech mis-recognitions.
If I have a test case like:
{
"text": "contact info for carole",
"intent": "ContactInfo",
"entities": [
{
"matchText": "carole",
"entityType": "builtin.personName"
}
]
}
And through speech transcription, we instead get something like:
{
"text": "contact info for carol",
"intent": "ContactInfo",
"entities": [
{
"matchText": "carol",
"entityType": "builtin.personName"
}
]
}
Currently, this is reported as a missing entity.
This issue is probably not needed. We can't effectively test ASR -> entity results unless we are testing an end-to-end system and the entities are fully disambiguated.
The correct approach here is to label the ASR output instead of the expected output.
I will file another issue to consider whether we want to remove speech -> LUIS result testing.