EISMANN-DEV

Hi ! while running it : `Wed Sep 6 22:39:05 2023 +---------------------------------------------------------------------------------------+ | NVIDIA-SMI 537.13 Driver Version: 537.13 CUDA Version: 12.2 | |-----------------------------------------+----------------------+----------------------+ | GPU Name TCC/WDDM | Bus-Id Disp.A...

can a oobabooga installation have an effect related to this ?

So I managed to fix it, first reinstalled oobabooga with cuda support (I dont know if it influenced localGPT), then completely reinstalled localgpt and its environment. EDIT : **I read...

So i ditched my RTX 4060 TI and moved to a RTX 4070, 8GB vs 12GB. I dont get any answer from this model, it just hangs : MODEL_ID =...

So I can confirm the models stopped working only because im now using run_localGPT_v2.py, when going back to run_localGPT.py its working again. Something for you @PromtEngineer ? thanks for the...

> @N1h1lv5 I hope with this new update, the issue is solved. Can you please confirm? The new run_localGPT.py is working, but some models still give empty answers, as you...

anyone ? from https://huggingface.co/TheBloke/Llama-2-13B-LoRA-Assemble-GPTQ `model_name_or_path = "TheBloke/Llama-2-13B-LoRA-Assemble-GPTQ" # To use a different branch, change revision # For example: revision="main" model = AutoModelForCausalLM.from_pretrained(model_name_or_path, device_map="auto", trust_remote_code=False, revision="main")` I tried to implement revision...

-removed- using now gguf models instead of GPTQ. sometimes getting empty answers. > Question: what is in the text ? > Answer: Enter a query:` I ask myself if the...

> @N1h1lv5 I am running into the same issue, empty answers. Still debugging it. It might be related to the promptTemplate. Will debug. You are the best ! thanks !

This allowed me to use cpu and gpu simultaniously with GGUF, for windows : set the environment variable properly : $Env:CMAKE_ARGS="-DLLAMA_CUBLAS=on" $Env:FORCE_CMAKE="1" check that it works echo $Env:CMAKE_ARGS uninstall previous...