`TypeError: fit() got an unexpected keyword argument 'callbacks'` with xgboost 2.1.0

[harupy]

trafficstars

Code

from flaml import AutoML
from flaml.automl.data import load_openml_dataset

# Download [houses dataset](https://www.openml.org/d/537) from OpenML. The task is to predict median price of the house in the region based on demographic composition and a state of housing market in the region.
X_train, X_test, y_train, y_test = load_openml_dataset(dataset_id=537, data_dir="./")

automl = AutoML()
settings = {
    "time_budget": 60,  # total running time in seconds
    "metric": "r2",  # primary metrics for regression can be chosen from: ['mae','mse','r2']
    "estimator_list": ["xgboost"],  # list of ML learners; we tune XGBoost in this example
    "task": "regression",  # task type
    "log_file_name": "houses_experiment.log",  # flaml log file
    "seed": 7654321,  # random seed
}
automl.fit(X_train=X_train, y_train=y_train, **settings)

Traceback

Traceback (most recent call last):
  File "//a.py", line 16, in <module>
    automl.fit(X_train=X_train, y_train=y_train, **settings)
  File "/usr/local/lib/python3.9/site-packages/flaml/automl/automl.py", line 1926, in fit
    self._search()
  File "/usr/local/lib/python3.9/site-packages/flaml/automl/automl.py", line 2483, in _search
    self._search_sequential()
  File "/usr/local/lib/python3.9/site-packages/flaml/automl/automl.py", line 2319, in _search_sequential
    analysis = tune.run(
  File "/usr/local/lib/python3.9/site-packages/flaml/tune/tune.py", line 814, in run
    result = evaluation_function(trial_to_run.config)
  File "/usr/local/lib/python3.9/site-packages/flaml/automl/state.py", line 304, in _compute_with_config_base
    ) = compute_estimator(
  File "/usr/local/lib/python3.9/site-packages/flaml/automl/ml.py", line 369, in compute_estimator
    val_loss, metric_for_logging, train_time, pred_time = task.evaluate_model_CV(
  File "/usr/local/lib/python3.9/site-packages/flaml/automl/task/generic_task.py", line 740, in evaluate_model_CV
    val_loss_i, metric_i, train_time_i, pred_time_i = get_val_loss(
  File "/usr/local/lib/python3.9/site-packages/flaml/automl/ml.py", line 494, in get_val_loss
    estimator.fit(X_train, y_train, budget=budget, free_mem_ratio=free_mem_ratio, **fit_kwargs)
  File "/usr/local/lib/python3.9/site-packages/flaml/automl/model.py", line 1661, in fit
    return super().fit(X_train, y_train, budget, free_mem_ratio, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/flaml/automl/model.py", line 1413, in fit
    self._fit(
  File "/usr/local/lib/python3.9/site-packages/flaml/automl/model.py", line 224, in _fit
    model.fit(X_train, y_train, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/xgboost/core.py", line 726, in inner_f
    return func(**kwargs)
TypeError: fit() got an unexpected keyword argument 'callbacks'

Package versions

> pip list
Package              Version
-------------------- -----------
argon2-cffi          23.1.0
argon2-cffi-bindings 21.2.0
certifi              2024.6.2
cffi                 1.16.0
charset-normalizer   3.3.2
FLAML                2.1.2  👈
idna                 3.7
joblib               1.4.2
liac-arff            2.5.0
minio                7.2.7
numpy                1.26.4
openml               0.14.2
pandas               2.2.2
pip                  23.0.1
pyarrow              16.1.0
pycparser            2.22
pycryptodome         3.20.0
python-dateutil      2.9.0.post0
pytz                 2024.1
requests             2.32.3
scikit-learn         1.5.0
scipy                1.13.1
setuptools           58.1.0
six                  1.16.0
threadpoolctl        3.5.0
typing_extensions    4.12.2
tzdata               2024.1
urllib3              2.2.2
wheel                0.42.0
xgboost              2.1.0  👈
xmltodict            0.13.0

Cause

The callbacks argument has been removed in xgboost 2.1.0.

https://github.com/dmlc/xgboost/releases/tag/v2.1.0:

For the Python package, eval_metric, early_stopping_rounds, and callbacks from now removed from the fit method in the sklearn interface. They were deprecated in 1.6. Use the parameters with the same name in constructors instead. (#9986)

[Programmer-RD-AI]

Hi,

I wanted to inform you that I am using the following package versions:

• FLAML: 2.1.2
• xgboost: 2.0.3

The code sample you gave me works correctly in my local environment. The issue might be related to the specific version of the xgboost library. FLAML might not have yet accounted for the latest changes in xgboost. I will investigate this further, but in the meantime, downgrading xgboost may resolve your issue.

Thank you.

[yxtay]

It is due to a breaking change in xgboost 2.1.0. Downgrading is the best workaround for now.

https://xgboost.readthedocs.io/en/latest/changes/v2.1.0.html#breaking-changes

https://github.com/dmlc/xgboost/pull/9986