localGPT

localGPT copied to clipboard

I am using CPU for execution. I am able to run `python ingest.py --device_type cpu`, but when I am using `python run_localGPT.py --device_type cpu` for executing chat bot in command...

vit0502

How to add a reranker model to run_localGPT.py?

Suiji12

auto-gptq and auto awq is breaking in requiremetns.txt

2

pip install -r requirements needs to remove 1. auto-gptq ,( this needs to be compiled from source) 2. auto awq and add 3. cmake (for running onnxruntime)

fanbyprinciple

Is there a way to change language for LLama 2?

3

Hi, i'm currently playing around with german language and documents and used the multilingual embedding models quite successfully. However when running LLama 2-Chat-7B i will always get answers in english...

NiklasClausius

I encountered a mistake when I executed run_localGPT_API.py

1

 And here is the webui：  how can I fix this bug？

Suiji12

Not using the GPU

8

Having 32 GB of GPU and 64GB of ram intel 17 13th gen processor its taking 2-4 min to response and not using GPU using llama-cpp-python==0.1.83 --no-cache-dir   what...

CODE-SAURABH

Keeps re-downloading entire model every new session

5

Hi, I have a problem with the program which keeps re-downloading the model for every new session. Does anyone knows a fix for this? (ps. I'm not a programmer please...

ChrisBNECBT

Support for Inter iRIS xe GPU

1

Dear Team, Please let me know how we can run localGPT by using intel iRIS GPU.

sadiqkavungal

model inference is pretty slow

30

```shell 2023-08-20 14:20:27,502 - INFO - run_localGPT.py:180 - Running on: cuda 2023-08-20 14:20:27,502 - INFO - run_localGPT.py:181 - Display Source Documents set to: True 2023-08-20 14:20:27,690 - INFO - SentenceTransformer.py:66...

shao-shuai

Support for Qwen-7B-Chat

5

Can we please support the [Qwen-7b-chat ](https://huggingface.co/Qwen/Qwen-7B-Chat)as one of the models using 4bit/8bit quantisation of the original models? Currently when I pass a query to localGPT, it returns be a...

manmax31