sklearn-evaluation icon indicating copy to clipboard operation
sklearn-evaluation copied to clipboard

Add Model Comparison Functionality to NotebookCollection

Open hcho1111 opened this issue 3 years ago • 2 comments

Model comparability with the plot argument of a NotebookCollection object could be improved.

At the moment, models must be cycled through in order to compare plots. See documentation here for example. Perhaps it would benefit to add additional functionality that allows side by side comparisons, similar to that described in the advanced matplotlib usage guide here.

Things to consider: limitations on the number of models that can be plotted simultaneously (formatting issues), whether plotting models simultaneously is affected by window size, the degree to which the quality of individual plots is affected (if axes/labels/symbols get too small, perhaps we could reevaluate the inputs to improve visibility/model quality).

hcho1111 avatar Sep 19 '22 20:09 hcho1111

this is a great idea. I actually released something like this, but only implemented it for confusion matrix: https://sklearn-evaluation.readthedocs.io/en/latest/user_guide/compare.html

the idea is that to compare two models you could do:

# plot for model one
plot_one = ...

# plot for model two
plot_two = ...


# compare models in a single plot
plot_one + plot_two

We could implement the same logic for the other plots we have, and then incorporate that into NotebookCollection. thoughts? Do you want to give this a try?

edublancas avatar Sep 27 '22 19:09 edublancas

@hcho1111 let's take this right after the blog is ready to publish

idomic avatar Sep 27 '22 19:09 idomic

we implemented some of these features as part of the plot module

edublancas avatar Dec 23 '22 15:12 edublancas