Zach Deane-Mayer comments

Results 138 comments of


                                            Zach Deane-Mayer

trafficstars

2.0.3 vs 4.0+ stacking results differ

Ok, so here's how you get the metrics from one model in a caretList to match ## 4.0.1 caret_6.0-94, caretEnsemble_4.0.1 | ncomp| RMSE| Rsquared| MAE| RMSESD| RsquaredSD| MAESD| |-----:|---------:|---------:|---------:|---------:|----------:|---------:| |...

2.0.3 vs 4.0+ stacking results differ

And here it is for repeated CV. They again match for one model in this simple scenario. ## 4.0.1 | ncomp| RMSE| Rsquared| MAE| RMSESD| RsquaredSD| MAESD| |-----:|---------:|---------:|---------:|---------:|----------:|---------:| | 1|...

2.0.3 vs 4.0+ stacking results differ

So you should be able to get the results from `caretList` to match with both versions. The `caretStack` results will continue to differ; however, due to the changes I made...

2.0.3 vs 4.0+ stacking results differ

The other option, if you wish to take it on, is to modify [extractBestPreds](https://github.com/zachmayer/caretEnsemble/blob/main/R/caretPredict.R#L152) and [caretList](https://github.com/zachmayer/caretEnsemble/blob/main/R/caretList.R). ## [extractBestPreds](https://github.com/zachmayer/caretEnsemble/blob/main/R/caretPredict.R#L152) Add an argument for `aggregate_resamples`, and set the default to True. Then...

2.0.3 vs 4.0+ stacking results differ

You're welcome— I'm glad to see people using my package!

2.0.3 vs 4.0+ stacking results differ

Better instructions to fix it here: https://github.com/zachmayer/caretEnsemble/issues/371

2.0.3 vs 4.0+ stacking results differ

This PR will add the new feature: https://github.com/zachmayer/caretEnsemble/pull/374 You will need to re-install from main once it merges, and then pass `aggregate_resamples=FALSE` to `caretList` and `caretStack`.

2.0.3 vs 4.0+ stacking results differ

please try the new version with aggregate_resamples=FALSE, which should allow you to do what you want