Tiago Melo

Right, in my case I would be trying to replicate a prediction interval despite being in a binary classification context - not a confidence interval in the "traditional" sense (eg....

Thank you very much for the comment and clarification. Indeed, I was mixing up CIs and PIs - and a CI is all I'm looking for in this case since...

Thank you for the discussion and your insights! I don't think I have the time/resources to formally conduct such a study, but will most definitely give the method a try....

Some quick and dirty results for a 95%CI, comparing the two methods with B = 10000 using a public classification dataset  I'm somewhat happy with this plot. The main...

Yes, the plot is exactly that and yes, the models are built by sampling the rows, not the columns. Here is the same plot, but now on the logit scale...

It's not possible to compare the Inducitve Venn-Abers Predictor intervals with the bootstrap-based ones, since one considers model calibration and the other doesn't. That being said, I'm backtracking and now...

Not sure what you mean.. even with deterministic models we would the same issue right? We could train 1000 Decision Trees that classify each sample as either 0 or 1,...

What's the true conditional probability in this case?

Does your model really have 2000 iterations with substantial improvements up until the end? Could you reduce 0.8 to something like 0.05? Maybe the 1600 iterations being selected always capture...

I was just now trying to reproduce what I initially posted and was getting to the same conclusion as you: all of them produced the same score. Apparently I had...