Ananderz
Ananderz
> Confirming that the following works: > > ``` > from ctransformers import AutoModelForCausalLM > llm = AutoModelForCausalLM.from_pretrained( > "TheBloke/Mistral-7B-Instruct-v0.1-GGUF", > model_file="mistral-7b-instruct-v0.1.Q2_K.gguf", > model_type="mistral", > gpu_layers=0) > ``` > >...
I tried with llama-2 and llama2 and read the ctransformers documentation and realized its just llama. The answer gets into a loop when using llama2 models: The telecom industry is...
Fixed it by implementing prompt template!
I finally figured out how to run GGML using GPU. I had the same issue as all of you where GPU would be at 0-1% use. I am on Windows...
Looking forward to this @marella . Is this still on your to do?
> Hi, yes. I was out of station with a slow internet for the past few days, so the progress has slowed down. I will start looking into the pending...
@marella been trying to implement this function on my own. I think I might almost be there and have it functional. The problem is that I can't get it to...
I did this @marella Here is my chains.py: ``` from typing import Any, Callable, Dict, Optional from langchain.chains import ConversationalRetrievalChain from langchain.memory import ConversationBufferMemory from .llms import get_llm from .vectorstores...
It repeats the question before it gives an answer and then the repeated question is just removed
Having dependency issues when I run install. Quite a lot of them. Tried manually installing all the dependencies but still not able to get it to run properly. Do you...