ludwig icon indicating copy to clipboard operation
ludwig copied to clipboard

Expose method to pick metrics for output features

Open arnavgarg1 opened this issue 3 years ago • 0 comments

Describe the use case In certain scenarios, not all metrics may actually be useful to compute for a given output feature type. A user may only be interested in a subset of the metrics that are relevant to their problem. Today, Ludwig computes all the metrics for a given output feature type on each epoch, and then again while evaluating and computing overall stats. This can be computationally expensive, particularly in cases where a user is running training and evaluation on a CPU.

Describe the solution you'd like It would be great to have a way to expose an optional metrics property for output features that takes in a list of metric names to be computed on each epoch and for overall evaluation. That way, a subset of all available metrics can be picked.

arnavgarg1 avatar Aug 02 '22 18:08 arnavgarg1