shinomakoi
shinomakoi
Good idea, I'll look into that
Yeah this is something I'd like to add at some point. I'll look into it
Maybe this works now. I can't test though cause only 8GB VRAM. LoRAs seem to work at least Download the LoRA: https://huggingface.co/kaiokendev/superhot-13b-8k-no-rlhf-test - and set the path in the app...
They should work here now with Exllama backend, if set context to 8192 and 'Compress pos embeddings' to 4
Hmm I'm not sure. llama-cpp-python just got updated, maybe see if it works with the updated version. I'll look into AutoGPTQ
Not by default, instructions are here https://github.com/abetlen/llama-cpp-python/#windows-remarks ``` $env:CMAKE_ARGS = "-DLLAMA_CUBLAS=on" $env:FORCE_CMAKE = 1 pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir ``` Should do it (also need CUDA toolkit installed https://developer.nvidia.com/cuda-11-8-0-download-archive)...
Oh this looks interesting. I'll add it as a backend. Doesn't seem to support quantized models though, hopefully soon
Put the 'sd-dreamer-main' folder in the 'stable-diffusion-main' folder. So the path to be the main.py file would then be 'stable-diffusion-main\sd-dreamer-main\main.py'. I'll improve the instructions and make installing easier