0xbitches

Experiencing the same issues here, regressing to 4.16.0 fixed it for me but 4.21.0 is broken.

Can we close this issue? It looks like the problems are resolved with earlier fixes?

@SoftologyPro The issue template literally asks you to search before posting, so even if you close the issue people will be able to find it.

Suggested fix for people encountering this bug: Add `device_map={'': 0}` to line 18 of `modules/LoRA.py`, inside `PeftModel.from_pretrained`. This forces Peft to load the model with GPU, and should work with...

Giving this one a bump. ChatGLM results are pretty convincing honestly. @sgsdxzy How did you manage it?

> increase your swap size. Tried on windows to change virtual memory, but it does not appear to help. Maybe I will have to settle for using third party programs...

@RazeLighter777 I mentioned in the first comment that it's the first thing I tried. Sadly it does not work (at least on windows it seems).

Yeah modules is listed in the list of requirements for the extension but it will conflict with the modules/ folder in the textgen webui directory. Please consider removing this in...

Using llama-30b-4bit-128g downloaded here: https://huggingface.co/Neko-Institute-of-Science/LLaMA-30B-4bit-128g This is getting ridiculous...

@chavinlo thank you for your work! Are you able to train the LORA on 13b (or potentially larger)? Also, since the loss stops decreasing after ~1 epoch, it might not...