framework icon indicating copy to clipboard operation
framework copied to clipboard

Confusion matrix throws exception

Open icb088 opened this issue 5 years ago • 3 comments

What would you like to submit? (put an 'x' inside the bracket that applies)

  • [X ] question
  • [ ] bug report
  • [ ] feature request

Issue description Hi all,

I need a little help from the experts...

In one of my projects I am using Accord.NET Machine Learning (C4.5) for predicting trading profits and the confusion matrix to calculate some parameters.

In some special cases - when the output class just contains one value (see below) - the estimation of the confusion matrix throws the following exception: image

The data table is structured as follows ({Current] is only used for process flow while adding rows):

[TBO] (int) | [StdDev] (double) | [Profit] (yes/no) | [Current] (yes/no)

Working example (output = yes, no):

45 | 0.123 | yes | no 31 | 0.456 | yes | no 78 | 0.543 | no | no 56 | 0.845 | yes | no

Example throwing the exception (output = yes):

45 | 0.123 | yes | no 31 | 0.456 | yes | no 78 | 0.543 | yes | no 56 | 0.845 | yes | no

The following code fragment is used for codification, learning and creating the confusion matrix:

codebook = new Codification(c45Data);

DecisionVariable[] attributes =
{
	new DecisionVariable("TBO", DecisionVariableKind.Continuous),
	new DecisionVariable("StdDev", DecisionVariableKind.Continuous)
};

int classCount = codebook["Profit"].NumberOfSymbols;

tree = new DecisionTree(attributes, classCount);
C45Learning c45 = new C45Learning(tree);

DataTable symbols = codebook.Apply(c45Data);
double[][] inputs = symbols.ToJagged("TBO", "StdDev");
int[] outputs = symbols.ToArray<int>("Profit");

DecisionTree decisionTree = c45.Learn(inputs, outputs);

DecisionSet rules = tree.ToRules();
string ruleText = rules.ToString(codebook, "Profit", System.Globalization.CultureInfo.InvariantCulture);

ConfusionMatrix confusionMatrix = ConfusionMatrix.Estimate(decisionTree, inputs, outputs);

The last line is the one that throws the exception!

Can someone please advise me how to fix this issue - any help is welcome!

Many thanks in advance and best regards, Christian

icb088 avatar Sep 22 '20 09:09 icb088

Hi! I slightly peeked into CM source and found this https://github.com/accord-net/framework/blob/840aba6e077dee29c5978c6b697bcb56fae70f9d/Sources/Accord.Statistics/Analysis/Performance/ConfusionMatrix.cs#L187

Constructor used in this scenario dont alllow anything but 2 differs. D.

davidrapan avatar Sep 28 '20 18:09 davidrapan

Hi David,

first of all many thanks for your reply and the given information!

Does this mean that this is a bug - as it doesn't match with the error message - and should be changed to

if (expectedSymbols.Length > 2)

Thanks for your help and best regards, Christian

Am Mo., 28. Sept. 2020 um 20:00 Uhr schrieb David Rapaň < [email protected]>:

Hi! I slightly peeked into CM source and found this

https://github.com/accord-net/framework/blob/840aba6e077dee29c5978c6b697bcb56fae70f9d/Sources/Accord.Statistics/Analysis/Performance/ConfusionMatrix.cs#L187 http://url

Constructor used in this scenario dont alllow anything but 2 differs. D.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/accord-net/framework/issues/2203#issuecomment-700191532, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHTY6P5M5DQ4AWXEUM3J6ETSIDFLLANCNFSM4RVRZWNQ .

icb088 avatar Sep 30 '20 08:09 icb088

Hi, yes i guess so.

davidrapan avatar Sep 30 '20 13:09 davidrapan