Teknium

> > > Maybe tangentially related, but @tloen curious why you might want to leave typos in the dataset (per [#32 (comment)](https://github.com/tloen/alpaca-lora/pull/32#issuecomment-1474454667)) > > > > > > Not my...

Are you talking about fine tuning or generating dataset? You use gpt api's to generate the dataset, it doesn't require running anything on a gpu cloud vps

> Hey @MarkSchmidty do you have a link for the 4-bit peft - i'd like to see those results. > > I think this is one of the few repos...

> > I've tried 7B full fine tune alpaca and a 7b LORA and I find the lora to be greatly lacking > > But was the LoRA created in...

> Im on 0.37.2 and it still occurrs.

happening for me right now on latest transformers and bnb 0.37.2..

can anyone try peft 0.2.0 like @cnbeining change in his repo referencing this issue

> WizardLM has no concept of follow-up instructions. There's no tag to indicate the start of a question. Its input is formatted like: > > ``` > If a car...

> @chintu619 Thanks for your kindly feedback. > > Yes, we also found this issue on this version of training data, Evol-Instruct would output some repetitive content which have already...

Yep this model is imo SOTA, even compared to 30B llama fine tune of OA dataset, so long as it's retrained without the OAI boilerplate disclaimers. Very impressive model though!