GaLore Finetuning Example

[Uxito-Ada]

Description

add an example of new fine-tuning algorithm on IPEX-LLM llama2 7b #nano

1. Why the change?

as above

2. User API changes

How to Run

python galore_finetuning.py --repo-id-or-model-path '/home/wangruonan/llm-models/Llama-2-7b-chat-hf' --data-path 'g-ronimo/oasst2_top4k_en' --output-dir './ipex-llm-galore'

3. Summary of the change

GaLore Finetuning Example

4. How to test?

• [ ] N/A
• [ ] Unit test
• [x] Application test
• [ ] Document test
• [ ] ...

5. New dependencies

galore-torch trl==0.8.1