caretEnsemble
caretEnsemble copied to clipboard
Improve prediction intervals and documentation
This is a four part task:
- [ ] Create a more robust prediction interval for
predict.caretEnsemble - [ ] Apply this method to
predict.caretStackto allow it to produce intervals - [ ] Document prediction interval calculation and its interpretation
- [ ] Create a fall-back for ensembles that include models that fail to produce valid predictions
I am not sure how to handle part 4 just yet.
Hmmm, part 4 is tricky.
Part 2 is tricky as well. Some caretStack models (e.g. glm) will have prediction intervals, and others (e.g. gbm) will not.
Yes, Part 2 is really tricky if we want to include uncertainty within each model into the equation. Currently on the caretEnsemble side it just incorporates the disagreement between models within the library -- which is not enough.
Also, the performance of predict in both cases is pretty slow :turtle: so I'll want to check into that too.
Sounds good!