torchtune icon indicating copy to clipboard operation
torchtune copied to clipboard

Recommendations for obtaining validation dataset loss after each epoch

Open dcsuka opened this issue 8 months ago • 3 comments

For finetuning using a custom dataset, message converter function, and csv column format, how do we obtain validation losses on a separate csv with the same format at the end of each epoch? Do we need to wait until after training to run on all the checkpointed files?

Also, how can we generate outputs using the same message converter function and tune run generate, using a csv file with a single row as input?

dcsuka avatar Jun 01 '24 00:06 dcsuka