lifelines icon indicating copy to clipboard operation
lifelines copied to clipboard

utils.restricted_mean_survival_time() give a IntegrationWarning for large time

Open joefresna opened this issue 5 years ago • 0 comments

If I execute the following code

T=[190372.8581024417, 26206.040634069566, 54788.08314669597, 361626.1303776556, 314971.8104097218, 866.4733997333499, 18585.392881633317, 35809.72231934092, 253525.13675280445, 353233.0363513172, 49252.04601481139, 110122.31000427713, 258009.5881614012, 10629.097863427462, 29659.075281545924, 71835.06294758174, 252253.11246278908, 192315.92332276434, 69978.27458929886, 15443.266585802974, 113900.06297292595, 11963.470998188805, 123070.60723489878, 2514.617669857113, 422309.05644795805, 166720.28350460617, 111031.66901518499, 82289.8117872133, 58134.95728103934, 36129.48573355823]
E = [True, True, True, True, True, True, True, True, True, True, True, True, True, True, True,True, True, 
       True, True, True, True, True, True, True, True, True, True, True, True, True]
best_model, best_aic_ = utils.find_best_parametric_model( T, E, scoring_method="AIC")
mean, variance = utils.restricted_mean_survival_time(best_model, return_variance=True)

The second function utils.restricted_mean_survival_time() has issues when T has very large numbers. It prints out the following warnings:

[126584.87687993835] [nan]
/usr/local/lib/python3.6/dist-packages/lifelines/utils/__init__.py:279: IntegrationWarning: The algorithm does not converge.  Roundoff error is detected
  in the extrapolation table.  It is assumed that the requested tolerance
  cannot be achieved, and that the returned result (if full_output = 1) is 
  the best which can be obtained.
  return quad(model.predict, 0, t)[0]
/usr/local/lib/python3.6/dist-packages/lifelines/utils/__init__.py:319: IntegrationWarning: The integral is probably divergent, or slowly convergent.
  return 2 * quad(lambda tau: (tau * model.predict(tau)), 0, t)[0]
/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:17: RuntimeWarning: invalid value encountered in sqrt

Strangely, this warning disappears when I reduce the values in T with, for instance,

T=[t/1000 for t in T]

joefresna avatar Aug 27 '20 17:08 joefresna