Trianing speed is super slow with your L-BFGS

[mhmdfiqrii]

Hi, Thank you for your post, it helps a lot. However when I tried to use it I found that it ios very slow even with GPU. With SGU and only one training sample it took 3minutes while with L-BFGS it couldn't reach convergence for 1 hour! Is there anything wrong?

P.s. The structure of network consists only three liner layers.

Best, Itsu

[dsanno]

Thank you for your report. I've reproduced the problem. I'm not familiar with L-BFGS, but I think history size of it (default: 100) is too large. Please try smaller history size like:

    optimizer = LBFGS(learning_rate, size=10)

[mhmdfiqrii]

Thank you for your reply. I will try.

Best, Itsu