pytorch-lr-finder icon indicating copy to clipboard operation
pytorch-lr-finder copied to clipboard

Published 20 hours ago • verified publisher icon davidtvs

Support pytorch's native amp

Open daniel-j-h opened this issue 5 years ago • 4 comments
trafficstars

Hey hey :wave:

PyTorch now has it's native amp module for a while now

https://pytorch.org/docs/stable/amp.html

it would be great to move to that, or at least prefer using it if it's available.

daniel-j-h avatar Nov 17 '20 09:11 daniel-j-h

Yeah, thanks for letting us know about this!

We had noticed that apex was going to be supported in PyTorch natively since v1.5, but didn't pay too much attention on its progress. But it seems torch.cuda.amp is stable enough now. I'll submit a new PR for it. 🤘

NaleRaphael avatar Nov 17 '20 11:11 NaleRaphael

Awesome, this was quick 🎉 🤗

On November 17, 2020 11:19:17 AM UTC, Nale Raphael [email protected] wrote:

Yeah, thanks for letting us know about this!>

We had noticed that apex was going to be supported in PyTorch natively since v1.5, but didn't pay too much attention on its progress. But it seems torch.cuda.amp is stable enough now. I'll submit a new PR for it. 🤘>

-- > You are receiving this because you authored the thread.> Reply to this email directly or view it on GitHub:> https://github.com/davidtvs/pytorch-lr-finder/issues/67#issuecomment-728862243

daniel-j-h avatar Nov 17 '20 11:11 daniel-j-h

Hey @daniel-j-h,

I would like to let you know that I have been busy with other things these weeks, so that the PR would be submitted lately. I've reviewed the difference between the implementation of apex.amp and torch.cuda.amp (v1.4 ~ v1.7) a few weeks ago, and some sketches of the changes to lr-finder had been done. But there are additional tests should be done to make sure it works for both apex.amp and torch.cuda.amp, hence that I have to take more time on it.

I'm sorry for any inconvenience resulted by this delay. I'll update this thread when the PR is ready. But if you have any further questions, please feel free to let me know. I might be free for answering them within a short time.

NaleRaphael avatar Dec 09 '20 15:12 NaleRaphael

All good, I think this is important but not urgent :hugs: Thank you for your transparency here! :bow:

daniel-j-h avatar Dec 09 '20 16:12 daniel-j-h

From what I can tell in the documentation, apex is still needed to take advantage of AMP with this library. Is that the case or is this issue completed?

carbocation avatar Aug 27 '23 04:08 carbocation

Closing, I believe https://github.com/davidtvs/pytorch-lr-finder/pull/91 addressed this issue

davidtvs avatar Feb 03 '24 10:02 davidtvs