Pavel Izmailov

Hi, this blogpost describes the SWA implementation from this repo: https://pytorch.org/blog/stochastic-weight-averaging-in-pytorch/

Hi @Balandat, looking into this right now. I believe the issue is that L-BFGS optimizer requires all the parameters to be on the same GPU (see second warning here: https://pytorch.org/docs/stable/optim.html#torch.optim.LBFGS)....

Hi @milliema I'd say you should do the same thing that is normally done with the batchnorm statistics in the end of parallel training, I imagine you are syncing the...

Hi @jizongFox thanks for the PR. I will be able to take a close look in a week or so. I don't know if we want to add `tqdm` in...

Hey @byan19! We used the code from our other project ([repo](https://github.com/timgaripov/dnn-mode-connectivity)) to make these visualizations. Please see [plane.py](https://github.com/timgaripov/dnn-mode-connectivity/blob/master/plane.py) for evaluating the necessary values of the loss and [plane_plot.py](https://github.com/timgaripov/dnn-mode-connectivity/blob/master/plane_plot.py) for making...

Hey @Haoxiang-Wang, @sanyalsunny111! I re-ran 5 checkpoints for Waterbirds and CelebA and uploaded them to this [google drive](https://drive.google.com/drive/folders/1OQ_oPPgxgK_7j_GCt71znyiRj6hqi_UW?usp=sharing). The results are the following: - Waterbirds: 92.0 ± 0.9 worst group...

Hi @sanyalsunny111, if you want to get the results for DFR_Val, you should be looking at the results under "DFR on Validation", and then `test_worst_acc`, which in your screenshot is...

Hi @AnabetsyR, sorry for the late reply! I have never used Torchbearer, from their github page it seems like it is no longer maintained, and the [successor is pytorch lightning](https://medium.com/pytorch/pytorch-frameworks-unite-torchbearer-joins-pytorch-lightning-c588e1e68c98)....

Happy to hear that you liked the paper :) Sounds great, I hope that solution works out!

Hi @luningsun, sorry for the late reply! I looked into it, and I believe there was some change to `hamiltorch`, so I get the same error. I managed to get...