axolotl
axolotl copied to clipboard
Add Documentation on how to perform Multi-Node Finetuning in Slurm
What piece of documentation is affected?
https://github.com/OpenAccess-AI-Collective/axolotl/blob/main/docs/multi-node.qmd
What part(s) of the article would you like to see updated?
Similar to the currently available documentation, it would be great to have a step-by-step tutorial on how to set up multiple nodes in a cluster orchestrated by Slurm (link). This would be useful for conducting FFT runs on relatively large clusters that are commonly managed by Slurm.
Expected Outcome: A few sample Slurm scripts and a detailed step-by-step tutorial.
Additional Information
No response
Acknowledgements
- [X] My issue title is concise, descriptive, and in title casing.
- [X] I have searched the existing issues to make sure this feature has not been requested yet.
- [X] I have provided enough information for the maintainers to understand and evaluate this request.