LinAlgError: Singular matrix

[yuenshingyan]

I am just trying out NGB and the LinAlgError occured. It seems the matrix has a determinant of zero, according to this post https://stackoverflow.com/questions/10326015/singular-matrix-issue-with-numpy. I have attached the datasets that I used.

Code

ngb = NGBClassifier(verbose=False)
ngb.fit(X_train, y_train)

Traceback

Complete error traceback

LinAlgError                               Traceback (most recent call last)
Input In [103], in <cell line: 5>()
     22 # NGBoost
     23 ngb = NGBClassifier(verbose=False)
---> 24 ngb.fit(X_train, y_train)
     25 y_proba = ngb.predict_proba(X_test)[:, 1]
     27 # Calculate EVs and backtest with differnt EV-rankings

File ~/anaconda3/lib/python3.9/site-packages/ngboost/ngboost.py:312, in NGBoost.fit(self, X, Y, X_val, Y_val, sample_weight, val_sample_weight, train_loss_monitor, val_loss_monitor, early_stopping_rounds)
    310 loss_list += [train_loss_monitor(D, Y_batch, weight_batch)]
    311 loss = loss_list[-1]
--> 312 grads = D.grad(Y_batch, natural=self.natural_gradient)
    314 proj_grad = self.fit_base(X_batch, grads, weight_batch)
    315 scale = self.line_search(proj_grad, P_batch, Y_batch, weight_batch)

File ~/anaconda3/lib/python3.9/site-packages/ngboost/scores.py:12, in Score.grad(self, Y, natural)
     10 if natural:
     11     metric = self.metric()
---> 12     grad = np.linalg.solve(metric, grad)
     13 return grad

File <__array_function__ internals>:5, in solve(*args, **kwargs)

File ~/anaconda3/lib/python3.9/site-packages/numpy/linalg/linalg.py:393, in solve(a, b)
    391 signature = 'DD->D' if isComplexType(t) else 'dd->d'
    392 extobj = get_linalg_error_extobj(_raise_linalgerror_singular)
--> 393 r = gufunc(a, b, signature=signature, extobj=extobj)
    395 return wrap(r.astype(result_t, copy=False))

File ~/anaconda3/lib/python3.9/site-packages/numpy/linalg/linalg.py:88, in _raise_linalgerror_singular(err, flag)
     87 def _raise_linalgerror_singular(err, flag):
---> 88     raise LinAlgError("Singular matrix")

LinAlgError: Singular matrix

Specs OS: Ubuntu 22.04.1 LTS x86_64 Host: G7 MD Kernel: 5.15.0-53-generic Uptime: 4 hours, 38 mins Packages: 2102 (dpkg), 19 (snap) Shell: bash 5.1.16 Resolution: 1920x1080 DE: GNOME 42.5 WM: Mutter Terminal: gnome-terminal CPU: 11th Gen Intel i7-11800H (16) @ GPU: NVIDIA GeForce RTX 3050 Ti Mobi GPU: Intel TigerLake-H GT1 [UHD Grap Memory: 10436MiB / 15780MiB

Datasets y_train.csv X_train.csv

[alejandroschuler]

try scaling each column of your predictor matrix so that values are between 0 and 1.

[yuenshingyan]

try scaling each column of your predictor matrix so that values are between 0 and 1. The same error still occur.

[BD-Sengoku]

try scaling each column of your predictor matrix so that values are between 0 and 1. The same error still occur.

I am also experiencing a similar error, please do you solve this problem

[tsiakmaki]

@BD-Sengoku, try setting the natural_gradient off ngb = NGBClassifier(... natural_gradient = False .. ) as suggested at https://github.com/stanfordmlgroup/ngboost/pull/320#issuecomment-1477322945