anujcb

I am getting this with Wizard-Vicuna-7B-Uncensored.ggmlv3.q4_0.bin, only when i send in embeddings from a vector db search results. inferences without retriever works fine with out this issue. I will try...

OK, i was able to make it work by reducing the number of docs to 1, any value above 1 throws the memory access violation 

I think, the issue maybe because of the special characters in the context. This was the context send to generate from llm.  i debugged it and intercepted the call...

binary_path: F:\ProgramData\Anaconda3\envs\scrapalot-research-assistant\lib\site-packages\bitsandbytes\cuda_setup\libbitsandbytes_cuda116.dll CUDA SETUP: Loading binary F:\ProgramData\Anaconda3\envs\scrapalot-research-assistant\lib\site-packages\bitsandbytes\cuda_setup\libbitsandbytes_cuda116.dll... ggml_init_cublas: found 1 CUDA devices: Device 0: NVIDIA GeForce GTX 1070, compute capability 6.1 INFO: Started server process [24636] INFO: Waiting for application...

> I think, the issue maybe because of the special characters in the context. This was the context send to generate from llm.  i debugged it and intercepted the...

> It would really help to diagnose this if you are able to reproduce it with one of the examples in this repository. If that's not possible, I would suggest...