alpaca-lora icon indicating copy to clipboard operation
alpaca-lora copied to clipboard

Published 20 hours ago • verified publisher icon tloen

trained LoRA on the WizardLM dataset

Open getorca opened this issue 1 year ago • 3 comments

I trained a LoRA on the WizardLM(https://github.com/nlpxucan/WizardLM) dataset.

You can check it out here, https://huggingface.co/winddude/wizardLM-LlaMA-LoRA-7B

Wasn't sure if I should do a pull request and add it to the readme as it's not alpaca. Let me know...

getorca avatar Apr 26 '23 21:04 getorca

Can you do a 13b the full 7b is already released ;o

teknium1 avatar Apr 27 '23 06:04 teknium1

I'll take a look, but I need to get a better handle on the hyperparams first. Plus I'm also looking at merging and cleaning a number of the instruct datasets, before another run.

getorca avatar Apr 27 '23 07:04 getorca

I'll take a look, but I need to get a better handle on the hyperparams first. Plus I'm also looking at merging and cleaning a number of the instruct datasets, before another run.

Right now I'm working on using gpt3.5-turbo to clean this dataset of "As a large language model" stuff. I'll lyk and share it if I can get it done

teknium1 avatar Apr 27 '23 19:04 teknium1