h2o-3 icon indicating copy to clipboard operation
h2o-3 copied to clipboard

Allow specifying `gainslift_bins` in `model_performance` call

Open tomasfryda opened this issue 1 year ago • 0 comments

Gainslift bins parameter influences the precision of the estimates that we get. It seems to me that it would make more sense to allow specifying gainslift_bins during the model performance call as we might have different needs for different analyses or we don't know in advance that we will want to have precise estimation of some gains/lift related statistics.

For example now if we want to get Kolmogorov-Smirnov statistic for some dataset we have to use:

perf = model.model_performance(data)
print(max(perf.gains_lift()["kolmogorov_smirnov"]))

And the quality of that estimate depends on the gainslift_bins value specified during training which seems impractical to me. So my proposal is to allow specifying gainslift_bins in the model_performace(...) call.

tomasfryda avatar Nov 21 '23 11:11 tomasfryda