xgboost

xgboost copied to clipboard

Hi,

thanks for the great package it almost always works like charm.

We recently update from python 3.10 to 3.11 and XGBoost from 1.7.6 to 2.0.3 and are now getting an error when using unscaled target values with objective=count:poisson

** Error**

Traceback (most recent call last):
  File "[...]/model/target/pymaven/envs/builder/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3508, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-5d9d54ebd639>", line 1, in <module>
    runfile('"[...]/model/src/main/python/scripts/minimal_example_xgboost.py', wdir='"[...]//model/src/main/python/scripts')
  File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_bundle/pydev_umd.py", line 198, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "[...]/model/src/main/python/scripts/minimal_example_xgboost.py", line 9, in <module>
    model.fit(X, y)
  File ""[...]/model/target/pymaven/envs/builder/lib/python3.11/site-packages/xgboost/core.py", line 730, in inner_f
    return func(**kwargs)
           ^^^^^^^^^^^^^^
  File ""[...]/model/target/pymaven/envs/builder/lib/python3.11/site-packages/xgboost/sklearn.py", line 1090, in fit
    self._Booster = train(
                    ^^^^^^
  File ""[...]/model/target/pymaven/envs/builder/lib/python3.11/site-packages/xgboost/core.py", line 730, in inner_f
    return func(**kwargs)
           ^^^^^^^^^^^^^^
  File ""[...]/model/target/pymaven/envs/builder/lib/python3.11/site-packages/xgboost/training.py", line 181, in train
    bst.update(dtrain, i, obj)
  File ""[...]/model/target/pymaven/envs/builder/lib/python3.11/site-packages/xgboost/core.py", line 2050, in update
    _check_call(
  File ""[...]/model/target/pymaven/envs/builder/lib/python3.11/site-packages/xgboost/core.py", line 282, in _check_call
    raise XGBoostError(py_str(_LIB.XGBGetLastError()))
xgboost.core.XGBoostError: [08:26:45] /Users/runner/work/xgboost/xgboost/src/learner.cc:440: Check failed: !std::isinf(mparam_.base_score): 
Stack trace:
  [bt] (0) 1   libxgboost.dylib                    0x0000000293038994 dmlc::LogMessageFatal::~LogMessageFatal() + 124
  [bt] (1) 2   libxgboost.dylib                    0x0000000293192e30 xgboost::LearnerConfiguration::InitBaseScore(xgboost::DMatrix const*) + 764
  [bt] (2) 3   libxgboost.dylib                    0x00000002931804bc xgboost::LearnerImpl::UpdateOneIter(int, std::__1::shared_ptr<xgboost::DMatrix>) + 140
  [bt] (3) 4   libxgboost.dylib                    0x0000000293058ea4 XGBoosterUpdateOneIter + 144
  [bt] (4) 5   libffi.8.dylib                      0x0000000101a7004c ffi_call_SYSV + 76
  [bt] (5) 6   libffi.8.dylib                      0x0000000101a6d74c ffi_call_int + 1208
  [bt] (6) 7   _ctypes.cpython-311-darwin.so       0x0000000101becbb4 _ctypes_callproc + 1208
  [bt] (7) 8   _ctypes.cpython-311-darwin.so       0x0000000101be6e34 PyCFuncPtr_call + 1188
  [bt] (8) 9   python3.11                          0x00000001009c41fc _PyObject_MakeTpCall + 332

System

• Darwin 21.6.0
• Architecture arm64
• Python 3.11.4 | packaged by conda-forge
• XGBoost 2.0.3

Reproduction Script

from xgboost import XGBRegressor
import numpy as np
model = XGBRegressor(objective='count:poisson')
n_examples = 1
magnitude=250
X = np.zeros((n_examples, 1))
y = np.ones((n_examples,)) * magnitude
model.fit(X, y)

When setting magnitude=100 it works. also when setting objective='reg:squarederror

Thank you very much.

Appears to be floating point overflow. Will investigate.

Could you please provide an update on this? I am also facing the same issue

I was able to mitigate the issue by setting base_score to sufficiently high number (I set it to 2000 as it's average of my label in my dataset).

This is not a solution, because if number of iterations is not sufficiently high, this may have negative impact on model quality.

I encounter the same problem

Same. Setting base_score does avoid the error, but for me it's not feasible since it degrades the model performance.

The original issue is resolved with https://github.com/dmlc/xgboost/pull/10298 . But still, the exponential family can be difficult to work with due to numeric overflows. The initial estimation is fixed, but gradient calculation still requires exp(y_predt), and if the target is large, exp will overflow. I don't see a general solution inside XGBoost.

However, if you know your count is large, you can shift the target away from zero and transform the target into a log scale before applying XGBoost.

Feel free to reopen if the issue persists.