[FEATURE] xgboost.cv returns boosters and out-of-fold predictions in python.

[titericz]

Xgboost.cv python returns only a list of metrics calculated for each fold. This function is great for tuning a model but lacks the ability to return the base models (boosters) and o.o.f. predictions once they are all available inside the function. Since R Xgboost.cv returns the base models would be great if python version does the same. Parameters like "return_booster" and "return_oof" would be great.

[giulianowurster]

class save_best_model(xgb.callback.TrainingCallback):
    def __init__(self, cvbooster):
        #super().__init__
        self._cvbooster = cvbooster
    def after_training(self, model):
        self._cvbooster[:] = [cvpack.bst for cvpack in model.cvfolds]
        return model

cvboosters = []

cv_results = xgb.cv(dtrain=data_dmatrix, params=params, nfold=3,
                    num_boost_round=50, early_stopping_rounds=10, 
                    metrics="rmse", as_pandas=True, seed=0,
                    callbacks=[save_best_model(cvboosters), ])

Here Hope this helps.