workflows icon indicating copy to clipboard operation
workflows copied to clipboard

plumbing post-processing for survival analysis

Open simonpcouch opened this issue 1 year ago • 1 comments

From Hannah in the linked discussion:

Okay so here are my thoughts on what plumbing post-processing for survival analysis would need.

Basic assumptions

  • We might tailor both predictions of survival time and predictions of survival probability.
  • If we tailor survival probability, we will do so at a specific single time point (hello eval_time our old friend).

I have not yet validated either assumption. Max, do you have a sense of whether they are valid?

For the implementation, this implies

  • How/where do we specify the predictions?
  • How/where do we specify eval_time?

Specifying the predictions

  • tailor() could take survival time .pred_time via the estimate argument -- the documentation already suggest that.
  • tailor() could take survival probability in the tibble form of .pred (containing .eval_time and .pred_survival) via the probabilities argument.

Specifying eval_time

  • [tailor] tailor() would drop time (the documentation for that as "the predicted event time" contradicts the documentation for estimate listing .pred_time) and instead take eval_time. The alternative would be to derive it from .pred within a given postprocessing operation and default to the first value if we need a single one. Either way, time would disappear.
  • [workflows] We currently can pass eval_time to predict() and augment() methods for workflows but since we currently don't need it earlier, there are not arguments for eval time to the fit() method for workflows or the specification via workflow().
    • If the post-processing operation doesn't need estimation, only predict.workflow() having an eval_time argument should be fine.
    • If it does need estimation, it could come from the tailor() specification.
  • [tune] The tuning functions have an eval_time argument which is required for dynamic and integrated survival metrics. If we need a single eval time to optimize for, we use the first one.
    • If a user includes a tailor in the workflow to be tuned, it would be nice to not make them specify eval_time twice: when making the tailor() and when calling the tuning function.
    • We could pass it through workflows (and make it gain an argument) or make a function to update the tailor. I lean towards updating the tailor.

Originally posted by @hfrick in https://github.com/tidymodels/workflows/pull/225#pullrequestreview-2035569467

simonpcouch avatar May 31 '24 14:05 simonpcouch