add fine-tuning code with lora support

[Muhtasham]

Closes #35 #41 #58

[Muhtasham]

@guoday

[kael53]

I need this to fine-tune

[BehsadRiemer]

I remember you telling me about Deepseek in May, props to you @Muhtasham 🏃‍♂️

[adamreed90]

@Muhtasham Should the prompt being built in build_instruction_prompt not match the example in the README.md:

<｜begin▁of▁sentence｜>User: {user_message_1}

Assistant: {assistant_message_1}<｜end▁of▁sentence｜>User: {user_message_2}

Assistant:

Apologies if I misunderstand.

[Muhtasham]

@adamreed90 Yes this PR is specifically for instruction fine-tuning, so the prompt format in build_instruction_prompt is intentionally different from the chat-based format:

As I pointed in the new README.MD, for training data preparation, please follow the Sample Dataset Format.

If you’re bringing a dataset in a different format (such as chat-based), it would require modification.