OneTrainer icon indicating copy to clipboard operation
OneTrainer copied to clipboard

Implement validation loss using FID scores and add corresponding documentation

Open IndigoDosSantos opened this issue 8 months ago • 4 comments

Description

This pull request introduces the implementation of validation loss using FID (Fréchet Inception Distance) scores and provides comprehensive documentation for this feature (for the wiki). The implementation includes:

  • Calculation of FID scores at regular intervals (currently after each epoch) using a separate validation image set
  • Storage of FID scores and generated images in the "epochs" folder within the workspace directory
  • Logging of FID scores to TensorBoard for visualization and monitoring

The accompanying documentation covers the following aspects:

  • Explanation of validation loss and its implementation using FID scores
  • Description of how validation loss complements other training metrics
  • Guidance on interpreting validation loss and its benefits for monitoring model performance
  • Details on the relationship between FID scores, model generalization, and overfitting
  • Recommendations for the size of the validation set, with 15% of the total dataset being a good middle ground
  • Implementation considerations for effectively utilizing validation loss, including:
  • Creating a separate validation image set
  • Configuring the "validation_images" concept in concepts.json
  • Storing FID scores and generated images in the "epochs" folder
  • Calculating FID scores after each epoch and logging them to TensorBoard

IndigoDosSantos avatar Jun 07 '24 17:06 IndigoDosSantos