cruise
cruise copied to clipboard
Add cross-validation in Trainers
So far, we evaluated the convergence by computing training error only. However, we can fall into overfitting (https://en.wikipedia.org/wiki/Overfitting), which can make a poor prediction for the unseen dataset.
Instead, to make sure that the model is trained well, we need to perform cross-validation. A simple way is to take a portion of training data set (e.g., select 10% of each worker's), and computes the error with the data set in each epoch (note that the cross-validation set should not be included in computing gradient).
Supporting n-fold cross validation in framework level could be another task to work on.