кɵɵѕнī

I also just compiled the wikipedia example on Windows with little issue, both with and without WSL2. I did 3 things: - Installed [CMake](https://cmake.org/download/), though this was my first experiment,...

I also added Kobold/text-generation-webui support so you can run Llama or whatever you want locally. I only tested it a bit, but it worked well back when I made it....

> Just to make sure does `LlamaForCausalLM` supports this feature already?(#22546 ) it seems that, still there are some errors when using `device_map="auto"` for this task. It does (#22329). I...

@KohakuBlueleaf that's a bit disingenuous, as you've changed quite a few other things 😉 I think I was able to reproduce what you were talking about on [your repo](https://github.com/KohakuBlueleaf/guanaco-lora) though....

This may be fixed by this PEFT [PR](https://github.com/huggingface/peft/pull/201)

Yeah, seems to be: https://github.com/TimDettmers/bitsandbytes/issues/162

For everyone dealing with this, it's because BitsAndBytes doesn't play nice with Trainer when it tries to do DataParallelism. We're not actually missing out, as DataParallelism is quite slow, and,...

> @kooshi Would this PR allow pipeline parallelism for inference on llama as well? Would it be possible to have a parallel sample for generate.py? That's a good question. I...

You're a legend. I got this running when you first posted it. Tomorrow I'm going to try to train 65b with this plus #131

Yeah, I'm currently playing with Laion [OIG](https://huggingface.co/datasets/laion/OIG) small-chip2, but I'm really looking forward to the result of their [Open-Assistant](https://github.com/LAION-AI/Open-Assistant) project, which will create a fully human generated, refined, and curated...