llm-foundry icon indicating copy to clipboard operation
llm-foundry copied to clipboard

Published 20 hours ago • verified publisher icon mosaicml

Adds precision to eval

Open mvpatel2000 opened this issue 2 years ago • 0 comments

Adds precision to eval. Sets MPT to bf16. For some reason, BF16 + FSDP requires mixed_precision: FULL. It works fine without FSDP. FP16 also works fine and gives basically the same numbers with FSDP on any setting.

mvpatel2000 avatar May 16 '23 20:05 mvpatel2000