ask-cli
ask-cli copied to clipboard
ask smapi get-result-for-nlu-evaluations result is missing the values for expected slot names
I'm submitting a...
[ ] Regression (a behavior that used to work and stopped working in a new release)
[x] Bug report
[ ] Performance issue
[ ] Feature request
[ ] Documentation issue or request
[ ] Other... Please describe:
Expected Behavior
The API response of ask smapi get-result-for-nlu-evaluations
includes the correct data for the property "expected" (same as defined in the NLU annotation).
Current Behavior
API response of ask smapi get-result-for-nlu-evaluations
has a wrong (missing) data in the property "expected". The slot name and value which is defined in the NLU annotation is missing. The test is still passing, but the actual and expected are not matching.
Same behavior is in the UI of the in NLU Evaluation Results, see attached Screenshot
{
"_links": {
...
}
},
"paginationContext": {
"totalCount": 1
},
"testCases": [
{
"actual": {
"intent": {
"confirmationStatus": "NONE",
"name": "HelloWorldIntent",
"slots": {
"name": {
"confirmationStatus": "NONE",
"name": "name",
"slotValue": {
"type": "Simple",
"value": "mike"
}
}
}
}
},
"expected": [
{
"intent": {
"name": "HelloWorldIntent",
"slots": {
"name": {}
}
}
}
],
"inputs": {
"utterance": "hello mike"
},
"status": "PASSED"
}
],
"totalFailed": 0
}
CLI Snapshot If applicable, add screenshots to help explain your problem.
Steps to Reproduce (for bugs)
- Create new sample skill.
- Go to Build -> Intents -> HelloWorldIntent -> add an utterance "hello {name}" with slot Type Amazon.Firstname
- Go to Build -> Intents -> Annotation Sets -> Create a NLU Annotation Set with Utterance: hello mike Expected Intent: HelloWorldIntent Expected slots: slot name=name, slot value=mike
- Go to Evaluate Model -> NLU Evaluation -> Select your Annotation and click Run evaluation
- Copy the evaluationId and do this cli request
ask smapi get-result-for-nlu-evaluations -s amzn1.ask.skill.xyz--evaluation-id xyz
Possible Solution
Your Environment and Context
- ask-cli version: 2.22.4
- Operating System and version: macOS 11.2.2
- Node.js version used for development: v12.16.1
- NPM version used for development: 6.13.4
Hi @arisen1145, thank you for creating this bug report. This seems like a problem with the SMAPI service, so we will try to investigate.