superchargez

> This seems to be dependent on the language, I see a similar effect with -l fi and several others. My understanding is that the problem originates from the training...

so, now BERT based models supported?

Is not it possible to spun up a llamacpp server and reference it in aici.sh? Will this work?

I also tried following: I downloaded the model from thebloke (huggingface) and put it in following code: ``` from ctransformers import AutoModelForCausalLM llm = AutoModelForCausalLM.from_pretrained('/path/to/wizardcoder.bin', model_type='starcoder') ```

This works in google colab, though only if you enable GPU. from ctransformers import AutoModelForCausalLM llm = AutoModelForCausalLM.from_pretrained('/path/to/wizardcoder.bin', model_type='starcoder') Is there a way to run it locallly, **WITHOUT** GPU please?

I got 16GB RAM and I am using debian 12. So, RAM should not be the issue here. I have it worked in free google colab which provides you 12GB...

File I used is (smallest one there): https://huggingface.co/TheBloke/WizardCoder-Guanaco-15B-V1.0-GGML/resolve/main/wizardcoder-guanaco-15b-v1.0.ggmlv1.q4_0.bin How much RAM should it use? (I think it can't run in colab, even though it appears that RAM consumption does not...

Tried to run it with kobold (in linux) and got following error: System Info: AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 |...

If I give is smaller context window then it may just work. How do I give it smaller context? ```python from ctransformers import AutoModelForCausalLM llm = AutoModelForCausalLM.from_pretrained('starcoder.bin', context_length=2000, model_type='starcoder') print(llm('What...

I have only one system with 16GB RAM, which currently has debian 12 on it. I'm not running on WSL, because I think it would require even more RAM. So...