scikit-posthocs
scikit-posthocs copied to clipboard
Alpha cannot be specified for critical difference diagram
Describe the bug
I haven't seen any way to provide alpha (0.05 or 0.1) for plotting critical difference diagram of Nemenyi-Friedman test. I'm trying to follow Demsar's paper link and this tutorial (which sadly uses Orange package), and:
- "The performance of two classifiers is significantly different if the corresponding average ranks differ by at least the critical difference"
- "critical values q_α are based on the Studentized range statistic divided by √2", and examples of such values are provided in the paper
- relevant scikit-posthocs code
So I get the p-values from the test. But looking at CD plotting function, we can see that it uses sign_array()
function with constant arguments, which is alpha=0.05
. So there is no way to plot CD diagram with any other alpha level. This should be configurable.