Blair Sadewitz
Blair Sadewitz
[kai_micromamba-fix.patch](https://github.com/henk717/KoboldAI/files/14070453/kai_micromamba-fix.patch) It was annoying me that I had to wait for the thing to do its thing twice, and I decided that was the last time I was *ever* gonna...
[email protected]:~/KoboldAI$ ./play.sh --model models/Aurora-Nights-103B-v1.0-5.0bpw-h6-exl2 --model_backend "ExLlama V2" --model_parameters help Colab Check: False, TPU: False INFO | __main__::732 - We loaded the following model backends: KoboldAI API KoboldAI Old Colab Method...
Hello, Today I tried dynamic rope scaling (factor 4) with the Holomax model, and it worked really well--used the entire 16k context without any obvious degradation in quality. Therefore, I'm...
Hello, There are a number of models I'd like to try which require this. I know that I asked you about this in the past, and IIRC you mentioned that...
Hello, I stumbled upon a huggingface downloader written in Go which uses git LFS. It's *fast*. I get considerably faster transfer speeds than with aria2 or git-lfs, and it doesn't...
### Your current environment ```text PyTorch version: 2.2.0+cu121 Is debug build: False CUDA used to build PyTorch: 12.1 ROCM used to build PyTorch: N/A OS: Ubuntu 22.04.4 LTS (x86_64) GCC...
### 🚀 The feature, motivation and pitch I don't know if it's feasible or worthwhile to merge [this](https://github.com/IBM/vllm/tree/9855b99502c7537db5ef018129e603650800ac46), as maybe the trees are too divergent, etc., but cherry-picking commits for...
### 🚀 The feature, motivation and pitch I use the koboldai webui a lot. Koboldcpp supports xtc (and dry rep pen, and maybe some other samplers) that I don't think...
Hello, Rather than instructing people to do 'sudo python3 setup.py install', I realized that, for preserving my sanity, it's a lot less trouble to use conda/mamba/micromamba (I use the last...