Add the support for XGBoost 3.0.0

Open c-bata opened this issue 8 months ago • 1 comments

Expected behavior

XGBoost 3.0.0 was released two weeks ago, causing the workflows to fail as follows.

https://pypi.org/project/xgboost/#history
https://github.com/optuna/optuna-integration/actions/runs/13981048377/job/39146195909

I will open a PR to add the version constraint, xgboost<=3.0.0, as a hotfix. This issue suggests fixing the above error and remove the version constraint.

Environment

Optuna version: main
Optuna Integration version: main
Python version: 3.9
OS: ubuntu-latest
(Optional) Other libraries and their versions: xgboost 3.0.0

Error messages, stack traces, or logs

_______________________ test_xgboost_pruning_callback_cv _______________________

    def test_xgboost_pruning_callback_cv() -> None:
        def objective(trial: optuna.trial.Trial) -> float:
            dtrain = xgb.DMatrix(np.ones((2, 1)), label=[1.0, 1.0])
            params = {
                "objective": "binary:logistic",
            }
    
            pruning_callback = XGBoostPruningCallback(trial, "test-logloss")
            xgb.cv(params, dtrain, callbacks=[pruning_callback], nfold=2)
            return 1.0
    
        study = optuna.create_study(pruner=DeterministicPruner(True))
>       study.optimize(objective, n_trials=1)

tests/xgboost/test_xgboost.py:73: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/optuna/study/study.py:475: in optimize
    _optimize(
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/optuna/study/_optimize.py:63: in _optimize
    _optimize_sequential(
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/optuna/study/_optimize.py:160: in _optimize_sequential
    frozen_trial = _run_trial(study, func, catch)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/optuna/study/_optimize.py:248: in _run_trial
    raise func_err
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/optuna/study/_optimize.py:197: in _run_trial
    value_or_values = func(trial)
tests/xgboost/test_xgboost.py:69: in objective
    xgb.cv(params, dtrain, callbacks=[pruning_callback], nfold=2)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/core.py:729: in inner_f
    return func(**kwargs)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/training.py:581: in cv
    booster.update(i, obj)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/training.py:229: in update
    fold.update(iteration, obj)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/training.py:215: in update
    self.bst.update(self.dtrain, iteration, fobj)
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/core.py:2246: in update
    _check_call(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

ret = -1

    def _check_call(ret: int) -> None:
        """Check the return value of C API call
    
        This function will raise exception when error occurs.
        Wrap every API call with this function
    
        Parameters
        ----------
        ret :
            return value from API calls
        """
        if ret != 0:
>           raise XGBoostError(py_str(_LIB.XGBGetLastError()))
E           xgboost.core.XGBoostError: [23:18:15] /workspace/src/objective/./regression_loss.h:69: Check failed: base_score > 0.0f && base_score < 1.0f: base_score must be in (0,1) for logistic loss, got: 1
E           Stack trace:
E             [bt] (0) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x2a6acc) [0x7fb07cea6acc]
E             [bt] (1) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0xed8329) [0x7fb07dad8329]
E             [bt] (2) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x681223) [0x7fb07d281223]
E             [bt] (3) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x6815ec) [0x7fb07d2815ec]
E             [bt] (4) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x68bb3b) [0x7fb07d28bb3b]
E             [bt] (5) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(XGBoosterUpdateOneIter+0x77) [0x7fb07cdb6ba7]
E             [bt] (6) /lib/x86_64-linux-gnu/libffi.so.8(+0x7b16) [0x7fb0b0825b16]
E             [bt] (7) /lib/x86_64-linux-gnu/libffi.so.8(+0x43ef) [0x7fb0b08223ef]
E             [bt] (8) /lib/x86_64-linux-gnu/libffi.so.8(ffi_call+0x12e) [0x7fb0b08250be]

/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/core.py:310: XGBoostError
----------------------------- Captured stderr call -----------------------------
[I 2025-03-20 23:18:15,587] A new study created in memory with name: no-name-fbb9dd4f-9283-4813-9952-b27d7124cc69
[W 2025-03-20 23:18:15,589] Trial 0 failed with parameters: {} because of the following error: XGBoostError('[23:18:15] /workspace/src/objective/./regression_loss.h:69: Check failed: base_score > 0.0f && base_score < 1.0f: base_score must be in (0,1) for logistic loss, got: 1\nStack trace:\n  [bt] (0) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x2a6acc) [0x7fb07cea6acc]\n  [bt] (1) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0xed8329) [0x7fb07dad8329]\n  [bt] (2) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x681223) [0x7fb07d281223]\n  [bt] (3) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x6815ec) [0x7fb07d2815ec]\n  [bt] (4) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x68bb3b) [0x7fb07d28bb3b]\n  [bt] (5) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(XGBoosterUpdateOneIter+0x77) [0x7fb07cdb6ba7]\n  [bt] (6) /lib/x86_64-linux-gnu/libffi.so.8(+0x7b16) [0x7fb0b0825b16]\n  [bt] (7) /lib/x86_64-linux-gnu/libffi.so.8(+0x43ef) [0x7fb0b08223ef]\n  [bt] (8) /lib/x86_64-linux-gnu/libffi.so.8(ffi_call+0x12e) [0x7fb0b08250be]\n\n').
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/optuna/study/_optimize.py", line 197, in _run_trial
    value_or_values = func(trial)
  File "/home/runner/work/optuna-integration/optuna-integration/tests/xgboost/test_xgboost.py", line 69, in objective
    xgb.cv(params, dtrain, callbacks=[pruning_callback], nfold=2)
  File "/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/core.py", line 729, in inner_f
    return func(**kwargs)
  File "/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/training.py", line 581, in cv
    booster.update(i, obj)
  File "/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/training.py", line 229, in update
    fold.update(iteration, obj)
  File "/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/training.py", line 215, in update
    self.bst.update(self.dtrain, iteration, fobj)
  File "/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/core.py", line 2246, in update
    _check_call(
  File "/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/core.py", line 310, in _check_call
    raise XGBoostError(py_str(_LIB.XGBGetLastError()))
xgboost.core.XGBoostError: [23:18:15] /workspace/src/objective/./regression_loss.h:69: Check failed: base_score > 0.0f && base_score < 1.0f: base_score must be in (0,1) for logistic loss, got: 1
Stack trace:
  [bt] (0) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x2a6acc) [0x7fb07cea6acc]
  [bt] (1) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0xed8329) [0x7fb07dad8329]
  [bt] (2) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x681223) [0x7fb07d281223]
  [bt] (3) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x6815ec) [0x7fb07d2815ec]
  [bt] (4) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(+0x68bb3b) [0x7fb07d28bb3b]
  [bt] (5) /opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/xgboost/lib/libxgboost.so(XGBoosterUpdateOneIter+0x77) [0x7fb07cdb6ba7]
  [bt] (6) /lib/x86_64-linux-gnu/libffi.so.8(+0x7b16) [0x7fb0b0825b16]
  [bt] (7) /lib/x86_64-linux-gnu/libffi.so.8(+0x43ef) [0x7fb0b08223ef]
  [bt] (8) /lib/x86_64-linux-gnu/libffi.so.8(ffi_call+0x12e) [0x7fb0b08250be]

Steps to reproduce

Please checkout the CI logs below: https://github.com/optuna/optuna-integration/actions/runs/13981048377/job/39146195909

Additional context (optional)

No response

Mar 28 '25 02:03 c-bata

@c-bata The failure is due to a genuine bug in XGBoost 3.0+: XGBoost fails if the training data contains all identical labels. See https://github.com/dmlc/xgboost/issues/11499

Jun 04 '25 17:06 hcho3