LLaMA-Factory icon indicating copy to clipboard operation
LLaMA-Factory copied to clipboard

[Update] loader.py , evaluate will run separate evaluations on each eval_dataset

Open SrWYG opened this issue 5 months ago • 1 comments

If you pass a dictionary with names of datasets as keys and datasets as values, evaluate will run separate evaluations on each dataset. This can be useful to monitor how training affects other datasets or simply to get a more fine-grained evaluation

seq2seqtrainner support eval_dataset as Dict.

What does this PR do?

Fixes # (issue)

Before submitting

  • [ ✅ ] Did you read the contributor guideline?
  • [ ✅ ] Did you write any new necessary tests?
    • I test it in alpacha format data,mode sft, model qwen2.5-7B-Instruct ,both single GPU and 2 GPU using fsdp.
    • the loss will be print and be logged in tensorboard run logs, which can be filtered by _loss in your tensorboard webUI.

SrWYG avatar Sep 24 '24 02:09 SrWYG