Improve runtime of confidence interval calculation for GlobalMetric

[matanor]

Computing of confidence intervals for GlobelMetric objects (here) requires the recalculation of the metric multiple times, depending on the n_resamples parameter. This recalculation may be costly in runtime for some metrics, specifically those that score the prediction with an independent evaluation model.

The runtime for this computation should be improved, for example by caching the inference results of such evaluation models.

from chiti :You can make 100 inference requests. Then take 100 or 1000 random samples with replacement from this 100 set, which will give you 100/1000 different metrics for free, as you simply reuse the inference results . If the original set of 100 samples is random, then the above will give you a pretty accurate estimation of the confidence intervals without having to run inference on more than the original 100 instances.