ConvergenceWarning while trainning

[medphisiker]

Hello, auto-sklearn works great for my. But during training I often get messages like this:

/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_stochastic_gradient.py:557: ConvergenceWarning: Maximum number of iteration reached before convergence. Consider increasing max_iter to improve the fit.
  ConvergenceWarning)

Sometimes these lines are repeated about 100-200 times one after the other.

Please tell me if there is any way to increase this 'max_iter' for stochastic_gradient using by auto-sklearn? I tried increasing the time by using "time_left_for_this_task" and "per_run_time_limit". But it didn't help.

[mfeurer]

Thanks for reporting these warnings. They are actually nothing to worry about. Auto-sklearn tries all kinds of configurations, and if one does not work (due to an algorithm not converging) it will look in other areas of the configuration space.

The reason why this happens so often is due to the way we train the SGD: we train it first for 2 iterations, and then iteratively double the amount of iterations until we end up with 1024 iterations. It's very likely then that after 2 iterations such a warning is emitted (and also after 4 etc).

I'm leaving this open so that we actually hide this warning.

[medphisiker]

I understand this is how your algorithm works, but sklearn doesn't know this and issues warnings. =) Thank you for your quick and complete response.

Probably, are you planning to write a popular science article on how your algorithm works? Something like this popular science article How We Implement Hyperband in Optuna. It would be very interesting to read ) And these cute hand-drawn graphics =)

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs for the next 7 days. Thank you for your contributions.