Uni-Core icon indicating copy to clipboard operation
Uni-Core copied to clipboard

Published 20 hours ago • verified publisher icon dptech-corp

Pytorch native optim

Open leasunhy opened this issue 7 months ago • 0 comments

  • Use torch.optim.AdamW as fallback Adam implementation.
  • Support selecting the fused versions of the optimizers (via --use-fused-optimizer).

Speed: custom_fused (only available for Adam) > fused > foreach

leasunhy avatar Jul 18 '24 10:07 leasunhy