SWA with Torchbearer

[AnabetsyR]

Hi! I am trying to implement SWA with Torchbearer but I'm having issues with the state etc. Could anybody point me in the direction of some resources to better understand how to use SWA with Torchbearer, particularly the Trial class?

Thank you in advance! Anabetsy

[izmailovpavel]

Hi @AnabetsyR, sorry for the late reply!

I have never used Torchbearer, from their github page it seems like it is no longer maintained, and the successor is pytorch lightning. Lightning actually has its own implementation of SWA.

Generally, you may also want to try using the native pytorch implementation of SWA, but I don't know how easy it is to use it with Torchbearer.

[AnabetsyR]

Hi @izmailovpavel! Thank you so much for taking the time to get back to me! I considered both the native Pytorch implementation and the Lightning one. But because the rest of my code uses some usefull callbacks in Torchbearer, I was trying to avoid a full change. I wrote the Torchbearer guys and one of them taught me how to pass the native Pytorch swa as a callback to my Torchbearer implementation. I'm testing it now but I'm hopeful.

P.S. I loved reading your paper on SWA.

Thank you so much again!

[izmailovpavel]

Happy to hear that you liked the paper :)

Sounds great, I hope that solution works out!