h2o-llmstudio icon indicating copy to clipboard operation
h2o-llmstudio copied to clipboard

[FEATURE] Re-score existing validation predictions

Open psinger opened this issue 1 year ago • 0 comments

🚀 Feature

Add a functionality to re-score the existing validation predictions of an experiment using a different (or even same) metric.

Motivation

I regularly run into the use-case that I want to re-score an existing experiment with a different metrics such as different GPT endpoint. As generation is very expensive, it would be helpful to allow to directly re-use the previous experiments outputs and only re-score.

psinger avatar Dec 21 '23 08:12 psinger