stanford_alpaca icon indicating copy to clipboard operation
stanford_alpaca copied to clipboard

Published 20 hours ago • verified publisher icon tatsu-lab

Deepspeed Training vs FSDP?

Open teknium1 opened this issue 1 year ago • 0 comments

Someone told me there is a deepspeed training option in the code, can I ask why its not the default? Do we know if it's far faster, and if so, how much faster at training Llama?

teknium1 avatar Apr 12 '23 11:04 teknium1