openfl icon indicating copy to clipboard operation
openfl copied to clipboard

[Feature Request] ability to configure criteria to save best model

Open kminhta opened this issue 7 months ago • 0 comments

Currently, the Task Runner will save the best model based on the accuracy score acquired during aggregated model validation. A higher accuracy score indicates better performance (ref). This is good practice in general, but is limited to metrics where higher score indicates better performance. There will be cases where a user will use a loss metric for validation. In this case, lower values indicate better performance and the Task Runner will not save the best model since it is looking for the highest value.

This configuration should be via the enabled via plan.yaml A natural location to consider would be the task_*.yaml (ref)

kminhta avatar May 13 '25 14:05 kminhta