workflows
workflows copied to clipboard
plumbing post-processing for survival analysis
From Hannah in the linked discussion:
Okay so here are my thoughts on what plumbing post-processing for survival analysis would need.
Basic assumptions
- We might tailor both predictions of survival time and predictions of survival probability.
- If we tailor survival probability, we will do so at a specific single time point (hello
eval_timeour old friend).
I have not yet validated either assumption. Max, do you have a sense of whether they are valid?
For the implementation, this implies
- How/where do we specify the predictions?
- How/where do we specify
eval_time?
Specifying the predictions
tailor()could take survival time.pred_timevia theestimateargument -- the documentation already suggest that.tailor()could take survival probability in the tibble form of.pred(containing.eval_timeand.pred_survival) via theprobabilitiesargument.
Specifying eval_time
- [tailor]
tailor()would droptime(the documentation for that as "the predicted event time" contradicts the documentation forestimatelisting.pred_time) and instead takeeval_time. The alternative would be to derive it from.predwithin a given postprocessingoperationand default to the first value if we need a single one. Either way,timewould disappear. - [workflows] We currently can pass
eval_timetopredict()andaugment()methods for workflows but since we currently don't need it earlier, there are not arguments for eval time to thefit()method for workflows or the specification viaworkflow().- If the post-processing operation doesn't need estimation, only
predict.workflow()having aneval_timeargument should be fine. - If it does need estimation, it could come from the
tailor()specification.
- If the post-processing operation doesn't need estimation, only
- [tune] The tuning functions have an
eval_timeargument which is required for dynamic and integrated survival metrics. If we need a single eval time to optimize for, we use the first one.- If a user includes a tailor in the workflow to be tuned, it would be nice to not make them specify
eval_timetwice: when making thetailor()and when calling the tuning function. - We could pass it through workflows (and make it gain an argument) or make a function to update the tailor. I lean towards updating the tailor.
- If a user includes a tailor in the workflow to be tuned, it would be nice to not make them specify
Originally posted by @hfrick in https://github.com/tidymodels/workflows/pull/225#pullrequestreview-2035569467