transformers icon indicating copy to clipboard operation
transformers copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

exclude fsdp from delay_optimizer_creation

Open eljandoubi opened this issue 1 year ago • 5 comments

What does this PR do?

It passes the model and the optimizer to accelerate.prepare in order to enable fp8 mixed precision, if any.

Fixes #34024

Who can review?

Library:

  • trainer: @muellerzr and @SunMarc

-->

eljandoubi avatar Oct 13 '24 21:10 eljandoubi