axolotl
axolotl copied to clipboard
Update multi-node.qmd
Title: Distributed Finetuning For Multi-Node with Axolotl and Deepspeed
Description: This PR introduces a comprehensive guide for setting up a distributed finetuning environment using Axolotl and Accelerate. The guide covers the following steps:
- Configuring SSH for passwordless access across multiple nodes
- Generating and exchanging public keys for secure communication
- Configuring Axolotl with shared settings and host files
- Configuring Accelerate for multi-node training with Deepspeed
- Running distributed finetuning using Accelerate