cruise icon indicating copy to clipboard operation
cruise copied to clipboard

Add cross-validation in Trainers

Open yunseong opened this issue 8 years ago • 1 comments

So far, we evaluated the convergence by computing training error only. However, we can fall into overfitting (https://en.wikipedia.org/wiki/Overfitting), which can make a poor prediction for the unseen dataset.

Instead, to make sure that the model is trained well, we need to perform cross-validation. A simple way is to take a portion of training data set (e.g., select 10% of each worker's), and computes the error with the data set in each epoch (note that the cross-validation set should not be included in computing gradient).

yunseong avatar Jan 04 '17 14:01 yunseong

Supporting n-fold cross validation in framework level could be another task to work on.

gyeongin avatar Jan 05 '17 07:01 gyeongin