peft icon indicating copy to clipboard operation
peft copied to clipboard

[WIP-docs] Accelerate scripts

Open stevhliu opened this issue 1 year ago • 1 comments

This PR explores what a separate section for the Accelerate integrations (DeepSpeed + FSDP) could look like. The main parts I wanted to focus on are:

  • generalize the DeepSpeed script a bit more so users can configure it with other settings besides ZeRO-3/Offload
  • parts of the script users can customize if they want to use a different model, dataset, or PEFT method
  • highlight and briefly explain what the Accelerate parts are doing

~If we think this makes sense, I'll also do the FSDP script~ done :)

stevhliu avatar Apr 21 '23 16:04 stevhliu

The documentation is not available anymore as the PR was closed or merged.