Ravindra Marella

Can you please try reinstalling `ctransformers` with CUDA enabled: ```sh pip uninstall ctransformers --yes CT_CUBLAS=1 pip install ctransformers --no-binary ctransformers ```

Did you notice any performance drop if you don't set `gpu_layers`? Recently llama.cpp added full GPU acceleration (https://github.com/ggerganov/llama.cpp/pull/1827) which is added to `ctransformers` in `0.2.9` today. Can you please try...

You should set `gpu_layers` also. I just wanted to see if there is any performance difference when you don't set `gpu_layers`. So you can add it back. ```yml ctransformers: config:...

Thanks @nilvaes for the explanation. I suggest simply looking at the response generation speed instead of the GPU usage numbers. Try out both the CPU and GPU configs and see...

Please run the following command and post the output: ```sh pip show ctransformers nvidia-cuda-runtime-cu12 nvidia-cublas-cu12 ``` Make sure you have installed the CUDA libraries using: ```sh pip install ctransformers[cuda] ```

1. Please note that this is not a production ready project and should not be deployed publicly. The main use case is to run it locally. 2. You can add...

I will look into this. If the performance is good, I will make it as default, otherwise will add a config option to enable this. It will also solve #22

Hi, yes. I was out of station with a slow internet for the past few days, so the progress has slowed down. I will start looking into the pending issues...

Great! Please share link to your repo/branch.

The `db` directory will be created in the same location from where you ran the `chatdocs add` command. So you can run `rmdir db` from where you previously ran the...