h2o-llmstudio
h2o-llmstudio copied to clipboard
[FEATURE] Re-score existing validation predictions
🚀 Feature
Add a functionality to re-score the existing validation predictions of an experiment using a different (or even same) metric.
Motivation
I regularly run into the use-case that I want to re-score an existing experiment with a different metrics such as different GPT endpoint. As generation is very expensive, it would be helpful to allow to directly re-use the previous experiments outputs and only re-score.