PyTorch-LBFGS

PyTorch-LBFGS copied to clipboard

updated line 257 to remove the warning: overload of add_ is deprecated

dpnshvrm

Hi Michael, I wanted to share a stochastic first order optimizer idea and implementation for L-BFGS, in case you might find it interesting or even practical in your research (congrats...

mbhynes

mbhynes

Encountering RuntimeError in CUTEst Problem: INDEF

8

Dheevatsa, have you seen any errors of the following sort? ``` Traceback (most recent call last): File "lbfgs_tests.py", line 114, in obj, grad, lr, backtracks, clos_evals, grad_evals, fail = optimizer.step(p,...

hjmshi

This question has been bothering me for a long time I wonder how to determine the direction of the gradient

ALittlething

Hello @hjmshi , Recently I stumbled upon [the Bits&Bytes wrapper](https://gist.github.com/TimDettmers/c4ffe346f095ee4481aa3d4b4ad2ffe0). Since then I can't help but wonder whether we could adapt this idea for the second order methods, namely, 8-bit...

Demirrr

How to avoid"Line search failed"?

3

I'm trying FullBatchLBFGS with wolfe line search on a fitting task of a small dataset. 1. Levenbert-Marquardt is giving me fairly accuracy. Can I expect LBFGS provide similar accuracy as...

duhd1993

RuntimeError: Expected object of type torch.FloatTensor but found type torch.DoubleTensor for argument #3 'other'

2

The error occurs in functions/LBFGS.py, line 854. I think this error comes from t. It becomes a double precision number with the process of convergence. I tried ``` if(F_new >...

jihaonew

Trying to accelerate LBFGS

1

Thanks for this implementation. Recently, I've been working on a package to use machine learning for chemistry problems where I use pytorch to train some models. I have been able...

muammar