deepxde icon indicating copy to clipboard operation
deepxde copied to clipboard

Published 20 hours ago • verified publisher icon lululxvi

Why L-BFGS performs different in tensorflow.compat.v1 and pytorch

Open gaohuizhang opened this issue 11 months ago • 3 comments

Hi,

I tried to train the model with L-BFGS after 15000 iterations of Adam, but I got different results from tensorflow.compat.v1 and PyTorch, even if I use the exact same code, just a different backend. This is the loss history using tensorflow.compat.v1 tensorflow This is the loss history using Pytorch pytorch

Does anyone have any idea why this happens? Does this mean we have to use TensorFlow if we want to use the L-BFGS optimizer?

Thanks a lot

gaohuizhang avatar Apr 01 '24 00:04 gaohuizhang

Hello, try to use the same seed for both cases dde.config.set_random_seed(1). Maybe the results are different due to different initialization

bakhtiyar-k avatar Apr 01 '24 10:04 bakhtiyar-k

Thanks for your advice, but I tried several times and got similar results. I don't think it is caused by randomness of initialization.

gaohuizhang avatar Apr 02 '24 17:04 gaohuizhang

Because LBFGS is implemented differently in both libraries. You find the link the the particular implementation (papers) in the source code of tf and torch.

praksharma avatar Apr 24 '24 10:04 praksharma