cd-diagram icon indicating copy to clipboard operation
cd-diagram copied to clipboard

Not getting cliques for two classifier which are not significantly different

Open BINAYKUMAR943 opened this issue 2 years ago • 1 comments

Below is the result when I am trying to compare four different classifiers.

['CoEye' 'TSF' 'cBOSS' 'Mr-SEQL'] CoEye 5.0 Mr-SEQL 61.0 TSF 8.0 cBOSS 10.0 dtype: float64 CoEye 3.279570 TSF 2.559140 cBOSS 2.537634 Mr-SEQL 1.623656 dtype: float64 ('CoEye', 'Mr-SEQL', 1.6071230220582003e-11, True) ('Mr-SEQL', 'TSF', 1.5905280411437273e-07, True) ('Mr-SEQL', 'cBOSS', 1.3264426790567074e-06, True) ('CoEye', 'TSF', 1.6515911035945563e-05, True) ('CoEye', 'cBOSS', 4.231130849948383e-05, True) ('TSF', 'cBOSS', 0.4035623277672452, False) Index(['CoEye', 'TSF', 'cBOSS', 'Mr-SEQL'], dtype='object') [1, 2]

From the above result, it is clear that classifiers TSF and cBOSS are not significantly different. But I am not getting any clique for them in the CD. Below is the CD diagram obtained.

cd-diagram

BINAYKUMAR943 avatar Dec 14 '21 20:12 BINAYKUMAR943

I am not quite sure, but can you try adding more classifiers since 4 is not enough maybe ? Since the test is also based on friedmann

Due to the assumption that the test statistic has a chi squared distribution, the p-value is only reliable for n > 10 and more than 6 repeated measurements.

hfawaz avatar Dec 15 '21 15:12 hfawaz