ctransformers
ctransformers copied to clipboard
marella
Python bindings for the Transformer models implemented in C/C++ using GGML library.
Hi there. Gguf now supports multimodal models. Is ctransformers supporting those already? If do, where can I find some docs to add this to lollms?
Thank you for this good and cool project. I was wondering if you could make a metal and rocm wheels and post them on pypi. This makes install easier since...
No matter what I try I can't set the context_length of a GPTQ model. It's overridden by ExLLAMA, which then sets the cache size and context_length whatever it set as...
I am trying to use a LLM model that can use GPU , model works fine with CPU but when with gpu it gives following error Error-Traceback (most recent call...
Small QOL change, adding a cache_dir argument to the input of from_pretrained. In addition, removes the model after test_model runs, this reduces clutter, but may slow down frequent testing. (This...
Retrieval Automated Generation of local docs built in, like gpt4all has.
OSError: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found
When I build Docker image with Apple M1 Silicon. The base image includes with GCC 12. And I got some build error as following. ``` In file included from /ctransformers/models/ggml/ggml.c:274:...
