cd-diagram
cd-diagram copied to clipboard
Not getting cliques for two classifier which are not significantly different
Below is the result when I am trying to compare four different classifiers.
['CoEye' 'TSF' 'cBOSS' 'Mr-SEQL'] CoEye 5.0 Mr-SEQL 61.0 TSF 8.0 cBOSS 10.0 dtype: float64 CoEye 3.279570 TSF 2.559140 cBOSS 2.537634 Mr-SEQL 1.623656 dtype: float64 ('CoEye', 'Mr-SEQL', 1.6071230220582003e-11, True) ('Mr-SEQL', 'TSF', 1.5905280411437273e-07, True) ('Mr-SEQL', 'cBOSS', 1.3264426790567074e-06, True) ('CoEye', 'TSF', 1.6515911035945563e-05, True) ('CoEye', 'cBOSS', 4.231130849948383e-05, True) ('TSF', 'cBOSS', 0.4035623277672452, False) Index(['CoEye', 'TSF', 'cBOSS', 'Mr-SEQL'], dtype='object') [1, 2]
From the above result, it is clear that classifiers TSF and cBOSS are not significantly different. But I am not getting any clique for them in the CD. Below is the CD diagram obtained.
I am not quite sure, but can you try adding more classifiers since 4 is not enough maybe ? Since the test is also based on friedmann
Due to the assumption that the test statistic has a chi squared distribution, the p-value is only reliable for n > 10 and more than 6 repeated measurements.