dmlc
Optuna Pruning in 3.0.0?
I'm currently trying to use Optuna with Pruning for hyperparameter optimization with XGBClassifier and running into a a few issues since the 2.1.4 version changes as it seems like?
My Setup:
- XGBoost: 3.0.0 (or your version)
- Optuna: 4.2.1
- Python: 3.11
The Problem:
As you know, callbacks (and early_stopping_rounds/eval_metric) were removed from XGBClassifier.fit() in v2.1+. This breaks the standard way of using optuna.integration.XGBoostPruningCallback directly in fit().
What I Found/Tried:
Interestingly, passing callbacks and early_stopping_rounds directly to the XGBClassifier constructor seems to work without throwing a TypeError:
# Inside Optuna objective
pruning_callback = SafeXGBoostPruningCallback(trial, "validation_1-auc") # My safe callback
print_callback = PrintIterationCallback(print_every_n=100)
model = xgb.XGBClassifier(
# ... other params ...
early_stopping_rounds=50,
eval_metric='auc',
callbacks=[pruning_callback, print_callback]
)
My custom PrintIterationCallback runs fine this way. However, when I include my SafeXGBoostPruningCallback (which just prevents duplicate trial.report() calls), Optuna still sometimes gives warnings like:
UserWarning: The reported value is ignored because this step X is already reported.
(Even with my custom callback trying to report only once per step, maybe there's still a timing issue?)
This makes me unsure if the pruning mechanism is actually engaging correctly when the callback is passed via the constructor, or if it's just "running" without properly interacting with Optuna's pruner due to these reporting conflicts.
Also, what happens if you use xgb.train() instead of xgb.XGBClassifier.fit() ? Optuna is using xgb.train() in their example: https://github.com/optuna/optuna-examples/blob/14697194c0f1e8948a046c7d14ebf4fddcce514f/xgboost/xgboost_integration.py#L50-L56
This breaks the standard way of using optuna.integration.XGBoostPruningCallback directly in fit().
This is not the "standard way". Use the constructor parameters or set_params instead.
