Fine-tuning with ``--run_validation False`` still calls ``get_custom_dataset`` and does not save the model

[lucas-ventura]

System Info

llama-recipes v0.0.3 torch 2.4.1

Information

• [X] The official example scripts
• [X] My own modified scripts

🐛 Describe the bug

I'm encountering two issues when using finetuning.py with a custom dataset and setting --run_validation False:

1. Unexpected validation dataset generation: The script still calls get_custom_dataset to generate validation data. This seems unintended, as the custom dataset does not always have a method to create a validation dataset. As a workaround, adding --test_split "train" to generate "validation" data avoids errors, but this feels counterproductive and isn't an ideal solution since the validation data isn't actually used.

2. Model not saved: The model does not get saved at the end of the training process. This is unexpected behavior as one would assume the model should be saved regardless of whether validation is performed or not.

Thank you!

Error logs

No error logs.

Expected behavior

When --run_validation is set to False, the script should not attempt to generate or require validation data.

The model should be saved at the end of the training process, even if validation is not performed.