axolotl icon indicating copy to clipboard operation
axolotl copied to clipboard

Add Documentation on how to perform Multi-Node Finetuning in Slurm

Open juvi21 opened this issue 8 months ago • 0 comments

What piece of documentation is affected?

https://github.com/OpenAccess-AI-Collective/axolotl/blob/main/docs/multi-node.qmd

What part(s) of the article would you like to see updated?

Similar to the currently available documentation, it would be great to have a step-by-step tutorial on how to set up multiple nodes in a cluster orchestrated by Slurm (link). This would be useful for conducting FFT runs on relatively large clusters that are commonly managed by Slurm.

Expected Outcome: A few sample Slurm scripts and a detailed step-by-step tutorial.

Additional Information

No response

Acknowledgements

  • [X] My issue title is concise, descriptive, and in title casing.
  • [X] I have searched the existing issues to make sure this feature has not been requested yet.
  • [X] I have provided enough information for the maintainers to understand and evaluate this request.

juvi21 avatar Jun 14 '24 13:06 juvi21