ctransformers
ctransformers copied to clipboard
marella
Python bindings for the Transformer models implemented in C/C++ using GGML library.
Hi, I tried the following code, but my kernel crashed and restarted, let me know how I should fix this, thanks! : ``` from ctransformers import AutoModelForCausalLM llm = AutoModelForCausalLM.from_pretrained("TheBloke/Llama-2-13B-chat-GGML",...
Sorry I'm not too familiar with either libraries but I was wondering if it would be possible to add trust_remote_code. I understand that remote code may be specific to the...
I use Yarn-Llama-2-13B-64K-GGUF and set n_ctx = 8192 in ooba webui, but when context exceeds 4096, it replies gibberish. how can i fix it? the response like "c [tO r...
I have Python 3.10, ctransformers 0.2.24 ( I tested with 0.2.24 too) The code ``` llm = AutoModelForCausalLM.from_pretrained('llama-2-7b-chat.ggmlv3.q2_K.bin', model_type='llama') ``` Sistematically I have error ``` Traceback (most recent call last):...
``` Sure, why not? What would you like to know? ``` why it can not stop at stop tokenizer, it will print out this token
Hello, I get a weired issue when serving using ctransformers. code: ```python model = AutoModelForCausalLM.from_pretrained( args.base_model, **config ) iterator: Generator = model.generate(gen_kwargs["inputs"]) for chat_chunk in iterator: new_text = model.detokenize(chat_chunk) print(new_text,...
Does `ctransformers` currently support running vision-language model? I'd like to run BLIP2 model from Salesforce hosted on Hugging Face - https://huggingface.co/Salesforce/blip2-opt-2.7b
Has anyone gotten this library running with Llama 2? I can use it for simple Question-Answering flows, but it seems to break down as the prompt gets more complicated or...
Make CTransformers compatible with 🤗 Transformers: - [x] Add Transformers-compatible model - [x] Add Transformers-compatible tokenizer See: - https://github.com/marella/ctransformers/issues/91#issuecomment-1685118509 - https://github.com/marella/ctransformers/issues/13#issuecomment-1597662836
Hi, i can run below code last week without problem but i got below error since some days ago (after upgrade ctransformers lib). I am unable to run ctransformers with...
