Add eval_set in fit?

[jmrichardson]

Hi, Thank you for this excellent work. I was wondering if you could add the ability to send eval_set as a fit_params in search.py? This will allow the fit method to do early stopping (which currently isn't possible using sklearn gridsearchcv). I updated the search.py code in line 578-580 marked by 3 hash marks below:

                    for train, test in cv.split(X, y, groups):
                        X_test = X.iloc[test.tolist()]  ###
                        y_test = y[test.tolist()] ###
                        fit_params = {"eval_set": (X_test, y_test)} ###
                        jobs.append(delayed(_fit_and_score)(
                            clone(base_estimator), X, y, scorers,
                            train, test, self.verbose, parameters,
                            fit_params=fit_params,
                            return_train_score=self.return_train_score,
                            return_n_test_samples=True,
                            return_times=True, return_parameters=True,
                            error_score=self.error_score
                        ))

I know the above is a hack which is why maybe you could figure out a better way to do something like this in your package?

WIth the above edit, I can tune catboost using the CV folds for early stopping. As opposed to gridsearchcv where you have to pass in a fixed test set.