axolotl icon indicating copy to clipboard operation
axolotl copied to clipboard

Update multi-node.qmd

Open shahdivax opened this issue 8 months ago • 3 comments

Title: Distributed Finetuning For Multi-Node with Axolotl and Deepspeed

Description: This PR introduces a comprehensive guide for setting up a distributed finetuning environment using Axolotl and Accelerate. The guide covers the following steps:

  1. Configuring SSH for passwordless access across multiple nodes
  2. Generating and exchanging public keys for secure communication
  3. Configuring Axolotl with shared settings and host files
  4. Configuring Accelerate for multi-node training with Deepspeed
  5. Running distributed finetuning using Accelerate

shahdivax avatar Jun 07 '24 06:06 shahdivax