KoboldAI-Client
KoboldAI-Client copied to clipboard
KoboldAI
You cannot run any of the models when using their Premium GPUs like NVIDIA A100, which gives this error: ``` Launching KoboldAI with the following options : python3 aiserver.py --model...
As KoboldAI can use a lot of RAM VRAM etc. It would be interesting if you could use it in a TUI mode since web browsers require a lot of...
Is there any option to disable html tags cleaning for the output text?
I'm cleaning up my C: drive because it's a 1TB drive and I put the models on a separate drive. Problem is, the model browser refuses to browse other drives,...
https://huggingface.co/Salesforce/codegen-2B-multi
https://huggingface.co/blog/hf-bitsandbytes-integration As I understand it, this would allow loading 6B models (minus context etc. of course) with 6GB VRAM.
Tested on a Steam Deck with HSA_OVERRIDE_GFX_VERSION=10.3.0 as env in the docker-compose.yml. Successfully loads and runs OPT up to 350m on the gpu, most layers on 1.3B and up to...
When calling the API even in the API docs page, if a function that makes a change such as changing model adding to story or world data or deleting last...
I have been using KoboldAI for a couple of months without issues. I have been updating it occasionaly (option 1, stable release) and never had any problems so far. Today...
