MOE
MOE copied to clipboard
improve gradient descent (and optimizer) 'communication'
Right now gradient descent doesn't offer much introspection into what it did & whether it succeeded:
- indicate whether convergence occurred (e.g., did the gradient drop below tolerance?)
- indicate the best objective function value found so far
- allow users to indicate a minimum best (like if I can't find a objective value better than 1.5, then I should indicate failure)
1, 2 3 are readily done by constructing an "IOContainer" for optimizers. This could have 3 fields--point, value, success. On input, point, value are the initial guess & minimum objective value. On output, point, value, success are the result & whether optimization succeeded.
This goes hand in hand with GH-91 (e.g., having a tolerance condition is what indicates whether convergence occurs).