Adam-experiments icon indicating copy to clipboard operation
Adam-experiments copied to clipboard

Published 20 hours ago • verified publisher icon sgugger

AdamW vs Adam

Open pgsrv opened this issue 4 years ago • 0 comments

Hi! According to the results in 'Appendix: Full results' (https://www.fast.ai/2018/07/02/adam-weight-decay/) is it correct that Adam is better for fine-tuning, while AdamW is better for training from scratch? Thanks!

pgsrv avatar Feb 16 '21 12:02 pgsrv