Shishir comments

Repositories
Issues
Comments

Results 2 comments of


                                            Shishir

very slow GPU compared with CPU

> I’m running docker on windows to use gptq model, response is slow though it is using 12GB GPU, what can be the reason, how to handle it ? Google...

very slow GPU compared with CPU

> !pip install --upgrade tensorrt !git clone https://github.com/PromtEngineer/localGPT.git %cd localGPT !pip install -r requirements.txt !python ingest.py --device_type cuda !python run_localGPT.py --device_type cuda Thanks, but that doesn't work anymore on T4...