ludwig
ludwig copied to clipboard
Expose method to pick metrics for output features
Describe the use case In certain scenarios, not all metrics may actually be useful to compute for a given output feature type. A user may only be interested in a subset of the metrics that are relevant to their problem. Today, Ludwig computes all the metrics for a given output feature type on each epoch, and then again while evaluating and computing overall stats. This can be computationally expensive, particularly in cases where a user is running training and evaluation on a CPU.
Describe the solution you'd like
It would be great to have a way to expose an optional metrics property for output features that takes in a list of metric names to be computed on each epoch and for overall evaluation. That way, a subset of all available metrics can be picked.