workflowsets icon indicating copy to clipboard operation
workflowsets copied to clipboard

Published 20 hours ago • verified publisher icon tidymodels

Include execution time as a metric for workflow sets

Open luisDVA opened this issue 3 years ago • 2 comments

Feature

This came up on Day 1 of the ML w/ tidymodels workshop at rstudioconf.

In some cases it may be useful to consider code execution time as a metric for ranking models in a set in addition to the other metrics. Davis mentioned that intermediate parsnip objects store execution time so this can be extracted and used to weight predictive power with running time. Thanks!

luisDVA avatar Jul 26 '22 03:07 luisDVA

This is an interesting idea! We'll come back to this after rstudio::conf. :)

simonpcouch avatar Jul 26 '22 13:07 simonpcouch

Related to tidymodels/tune#248! Some work starting to come together here.

simonpcouch avatar Dec 19 '22 15:12 simonpcouch