tick
tick copied to clipboard
Improvement of SGD and base classes for stochastic solvers
Currently SGD is very monolithic and cannot be tuned enough. More flexibility in the tuning of learning rates and Polyak Ruppert average should be implemented (easy). Handling of sparse features as well (easy). This could also be done by adding base classes, since for now there is only StoSolver, while some more concept could be mutualized through solvers (variance reduction, averaging, etc...)