Ashok RS

@PromtEngineer please share your email or let me know where can I find it. yes. I saw the updated code. The support for GPT quantized model , the API, and...

@PromtEngineer Any luck pushing the updates to contant(s).py ? I am waiting for it to do my little retrofit.

@PromtEngineer I managed to get the basic **gradio** based capability working. I am not used to following the best practices to update my new set of code in your "**localGPT**"...

@ashleykleynhans I agree. I came across the same issue of "I don't know" more than 90% of the time. I am using a GPU with 24GB RAM. I was thinking...

I tried these two models as both these models gave me "out of memory" errors.. #model_id = "TheBloke/Wizard-Vicuna-13B-Uncensored-HF" # Gives out of memory error #model_id = "TheBloke/Vigogne-Instruct-13B-HF" # Gives out...

@Tchekda Thanks. If I add this parameter “max_memory={0: "15GB"}” to “AutoModelForCausalLM.from_pretrained(“ function, I can load and and run a 13B parameter model onto a GPU with 24GB Memory?

@Tchekda sure. I will try and let you know.

@Tchekda I tried today. **Scenario:** FYI ==> My GIT pull was on 6-June-2023. There are updates to "localGPT" I didn't take it yet. My GPU instance has 32Gb main RAM...

Oh.. Well! I asked my previous to bard.g.co It responded me as follows: Open the code and make this change to include the parameter 'offload_folder="/tmp/offload"' and the line after this...

After giving the query string, I hit the out of memory error again: (lenv) root@ip-172-30-1-74:/sw/localGPT# python3 run_localGPT.py Running on: cuda load INSTRUCTOR_Transformer max_seq_length 512 Using embedded DuckDB with persistence: data...