Allow model to be captured during training

[mjmckp]

When tuning the SVM parameters, ideally it would be possible to capture the model during training, so that you end up training the model once up to N iterations, but end up with M models, where each of the M models represents the model as it was at some number of iterations < N. Without this feature, it is necessary to train the model M times with different values of max_iter to get the same result.

A simple way to do this would be to allow a trained model to resume training, so that one could do the following:

• create a model
• train it N0 iterations
• save the model to file
• continue training on the same model another N1 iterations
• (repeat until total number of iterations is N)

Does this seem feasible?

[zeyiwen]

This is doable, but it seems not a widely used approach in the community. We are often interested in the "optimal" SVM when the training terminates. That is the training can't make any progress (i.e., ideally reach the global optimum, or maximum iteration when training is too long).

Would you give some insights why tuning the maximum iteration is necessary? We usually only tune C and the kernel parameters.