scikit-survival icon indicating copy to clipboard operation
scikit-survival copied to clipboard

Sample weights support in CoxnetSurvivalAnalysis and concordance_index_censored

Open hermidalc opened this issue 4 years ago • 1 comments

CoxnetSurvivalAnalysis and concordance_index_censored (or model score) currently do not support sample weights and this would be very useful to have when calling fit and score using these methods.

This is something GLMNET has and, with respect to Coxnet, the extension to the partial log-likelihood calculation to support sample weights is described in the paper in Section 2.5 https://web.stanford.edu/~hastie/Papers/v39i05.pdf.

I don’t know if the current version of GLMNET does more with sample weights in the Coxnet algorithm than what is described above in the paper. I have a hard time reading FORTRAN, but I believe the sample weights are the w variable here https://github.com/cran/glmnet/blob/8d764f2a609e8bfe0c32c9eabbd7e8c04c9394f6/src/glmnet5dpclean.f#L3543

You can also see how sample weights are applied to the c-index calculation here https://github.com/cran/glmnet/blob/8d764f2a609e8bfe0c32c9eabbd7e8c04c9394f6/R/Cindex.R#L37

Support for sample weights is very useful since it is quite common to have repeated measures, for example in biological datasets. My current workaround is I wrote a custom scikit-learn compatible CV iterator that takes a random sample from each group of repeated measures before producing folds or splits.

hermidalc avatar May 26 '20 23:05 hermidalc

At least for concordance_index_censored you already wrote support for sample weights it’s just not exposed in the API and all weights are set to 1

https://github.com/sebp/scikit-survival/blob/b71ba669f6069e88bea497a786f1ff7ac5757946/sksurv/metrics.py#L85

hermidalc avatar May 26 '20 23:05 hermidalc