MOE icon indicating copy to clipboard operation
MOE copied to clipboard

improve gradient descent (and optimizer) 'communication'

Open suntzu86 opened this issue 10 years ago • 0 comments

Right now gradient descent doesn't offer much introspection into what it did & whether it succeeded:

  1. indicate whether convergence occurred (e.g., did the gradient drop below tolerance?)
  2. indicate the best objective function value found so far
  3. allow users to indicate a minimum best (like if I can't find a objective value better than 1.5, then I should indicate failure)

1, 2 3 are readily done by constructing an "IOContainer" for optimizers. This could have 3 fields--point, value, success. On input, point, value are the initial guess & minimum objective value. On output, point, value, success are the result & whether optimization succeeded.

This goes hand in hand with GH-91 (e.g., having a tolerance condition is what indicates whether convergence occurs).

suntzu86 avatar May 02 '14 22:05 suntzu86