autorank icon indicating copy to clipboard operation
autorank copied to clipboard

Published 20 hours ago • verified publisher icon sherbold

Plot error

Open gerald4 opened this issue 2 years ago • 3 comments

Hi! I ran autorank to rank algorithms and the last sentence of the result is: "Based on the post-hoc Nemenyi test, we assume that there are no significant differences within the following groups: sobel and random; random and guidedbackprop; guidedbackprop and smoothgrad; smoothgrad and rise; rise, scorecam, and layercam. All other differences are significant.". However, this does not reflect on the plot (see figure). It seems that the plot function has a problem plot .

gerald4 avatar Feb 07 '22 10:02 gerald4

Could you provide data to replicate this?

sherbold avatar Feb 07 '22 10:02 sherbold

I just negated reverse in the call to get_sorted_rank_groups and it worked for my case... not sure if in general.

vnmabus avatar Jul 01 '22 14:07 vnmabus

The issue seems to be that the sorting from one of the used libraries is unreliable or was changed at some point. At least this is what it looks like based on what @AlexandreAbraham found in #11. When I have time (so definitively after the current lecture period), I will look at rewriting that part to not assume sorting behavior, hoping that this would then be fixed.

sherbold avatar Jul 02 '22 08:07 sherbold