[RFC][DOCS] Recipe [DOCS] ([DOC]umentation)

[SalmanMohammadi]

What is the purpose of this PR? Is it to

• [x] add a new feature
• [ ] fix a bug
• [x] update tests and/or documentation
• [ ] other (please add here)

Let's map out a user journey here:

I'm a l33t gamer with a 2xRTX 4090 battlestation. I've just beat Cyberpunk (on max settings, mind you), and now I'd like to get into fine-tuning LLMs. I'm excited about the latest release of my favourite animal-based LLM, Llama3.1. What do I want to do? Not 100% sure - fine-tuning an LLM for my Discord? The LLama 3.1 documentation has helped me find ✨torchtune✨! What can I do with it?? How can I quickly discover this? Oh, cool, there's a quick tutorial for fine-tuning Llama3? And it uses LoRA? Great! And there's 🔥🔥🔥 documentation on customizing my own datasets? Amazing!! Now, what else can I do with this LoRA "recipe"? Isn't there a page where I can understand what all these parameters mean, at once? ... Well, what other recipes are there? What else can I do with ✨torchtune✨? ...

Right now, don't have a clear way to communicate to our users which recipes we support, and how to quickly configure them. Documentation for recipes is:

• Hidden inside recipe files. Recipe-specific features are listed amongst features common to all recipes. This makes the documentation a bit difficult to parse.
• Hidden inside our config files, which are also replicated across our configs. Our config files contain some of the more crucial details when using recipes; which commands to run, which levers to pull.

I want to maximise the surface area of the features we expose in torchtune. Our design philosophy keeps things flat and modular, users can swap out components, models, and datasets freely. I wish for the ML PhD and the l33t gamer to be able to discover what we offer, and how to use it, with equal ease.

My contribution addresses this in the following ways:

• I propose a glossary of features which are common across all of our recipes, and also common amongst recipes with specialised fine-tuning features such as PEFT or FSDP(2).
• I propose a simple recipe documentation template. This template uses the commands we'd usually place inside the config files to allow users to quickly get started with the recipe. This template then includes copy-and-paste text for lists of features which we commonly expose in recipes - these link to relevant sections in the glossary above.
• These recipes are then simply indexed in the recipe overview.

I provide two examples in this PR; documentation for LoRA single device, and for QAT distributed. I'd like to put an issue up for documenting additional recipes so other people may help out here - it'll be a good first issue for many contributors..

There's also a couple things missing from the memory glossary; FSDP/FSDP2 (and maybe something else?), I'll also put issues up for this.

Test plan

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣀⣤⣤⣤⣤⣴⣤⣤⣄⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⣀⣴⣾⠿⠛⠋⠉⠁⠀⠀⠀⠈⠙⠻⢷⣦⡀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⣤⣾⡿⠋⠁⠀⣠⣶⣿⡿⢿⣷⣦⡀⠀⠀⠀⠙⠿⣦⣀⠀⠀⠀⠀ ⠀⠀⢀⣴⣿⡿⠋⠀⠀⢀⣼⣿⣿⣿⣶⣿⣾⣽⣿⡆⠀⠀⠀⠀⢻⣿⣷⣶⣄⠀ ⠀⣴⣿⣿⠋⠀⠀⠀⠀⠸⣿⣿⣿⣿⣯⣿⣿⣿⣿⣿⠀⠀⠀⠐⡄⡌⢻⣿⣿⡷ ⢸⣿⣿⠃⢂⡋⠄⠀⠀⠀⢿⣿⣿⣿⣿⣿⣯⣿⣿⠏⠀⠀⠀⠀⢦⣷⣿⠿⠛⠁ ⠀⠙⠿⢾⣤⡈⠙⠂⢤⢀⠀⠙⠿⢿⣿⣿⡿⠟⠁⠀⣀⣀⣤⣶⠟⠋⠁⠀⠀⠀ ⠀⠀⠀⠀⠈⠙⠿⣾⣠⣆⣅⣀⣠⣄⣤⣴⣶⣾⣽⢿⠿⠟⠋⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠉⠙⠛⠛⠙⠋⠉⠉⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀