rasa icon indicating copy to clipboard operation
rasa copied to clipboard

`rasa test core` does not generate valid confusion matrix for action predictions

Open amn41 opened this issue 2 years ago • 0 comments

Rasa Open Source version

3.1.0

Rasa SDK version

No response

Python version

3.7

What operating system are you using?

OSX

What happened?

steps to reproduce

  1. create project: rasa init
  2. train model: rasa train
  3. run core evaluation: rasa test core

expected result

results/TEDPolicy_confusion_matrix.png contains a confusion matrix for the TEDPolicy.

observed result

console output appears to show valid confusion matrix:

2022-07-19 10:21:36 INFO     rasa.utils.plotting  - Confusion matrix, without normalization: 
[[ 0  0  0  0  0  0  0]
 [ 0 13  0  0  0  0  0]
 [ 0  0  2  0  0  0  0]
 [ 0  0  0  2  0  0  0]
 [ 1  0  0  0  0  0  0]
 [ 0  0  0  0  0  8  0]
 [ 0  0  0  0  0  0  2]]

results/TEDPolicy_confusion_matrix.png is an empty plot with the title "Entity Confusion Matrix"

Command / Request

No response

Relevant log output

No response

amn41 avatar Jul 19 '22 08:07 amn41