auto-sklearn icon indicating copy to clipboard operation
auto-sklearn copied to clipboard

[Question] cross validation strategy/individual cross validation results?

Open quietscientist opened this issue 1 year ago • 0 comments

Is there a way to see 1) what cross validation strategy was selected and 2) what the individual cross validation results for a given model?

I am training a classifier model as follows:

automl = AutoSklearn2Classifier( ensemble_size=1, dataset_compression=False, allow_string_features=False, time_left_for_this_task=1200, per_run_time_limit=30,
metric=autosklearn.metrics.balanced_accuracy, delete_tmp_folder_after_terminate=False, memory_limit=None, disable_evaluator_output=False, )

After training, I fit the model with automl.fit(X_train, y_train) and refitting with automl.refit(X_train.copy(),y_train.copy()).

I can see the "mean test score" in automl.cv_results, but can't find any info related to individual cv results that were used to compute that mean. I also tried using AutoSklearn 1, specifying the cv strategy and the number of folds, but can still only find the mean in the cv_result.

Any advice on where to look/how to get that information?

Thanks!

quietscientist avatar Apr 21 '24 20:04 quietscientist