h2o-llmstudio icon indicating copy to clipboard operation
h2o-llmstudio copied to clipboard

Compare Zero-Epoch Prediction with Fine-Tuned Prediction as well as Validaiton Score Comparison

Open meganjkurka opened this issue 10 months ago • 0 comments

🚀 Feature

In the Validation Prediction Insights, include the Zero-Epoch Response (i.e. how would the original foundation model response) as well as the BLEU (or other validation score) on the Zero-Epoch Response. This will show the user what the original foundation model would say vs their fine-tuned model as well as the validation performance improvement from fine-tuning.

Motivation

This will allow the user to quickly see the value of their fine-tuning exercise.

meganjkurka avatar May 03 '24 16:05 meganjkurka