h2o-llmstudio
h2o-llmstudio copied to clipboard
Compare Zero-Epoch Prediction with Fine-Tuned Prediction as well as Validaiton Score Comparison
🚀 Feature
In the Validation Prediction Insights, include the Zero-Epoch Response (i.e. how would the original foundation model response) as well as the BLEU (or other validation score) on the Zero-Epoch Response. This will show the user what the original foundation model would say vs their fine-tuned model as well as the validation performance improvement from fine-tuning.
Motivation
This will allow the user to quickly see the value of their fine-tuning exercise.