johnbrisbin

In your README.md section, it looks like you need a: cd llama-cpp-python just before the: $Env:CMAKE_ARGS="-DLLAMA_CUBLAS=on"; $Env:FORCE_CMAKE=1; py ./setup.py install else setup.py is not found. The build that was to...

@maozdemir, thanks for responding. Looks like you might be in a different time zone. > > 1. Rather than scraping nvidia-smi, have you considered using pycuda? It is simple to...

I encountered another issue: The version of torch required is greater than the one required for non-CUDA support. This was a 2+ version to support Cuda 11.8.

> @johnbrisbin can you use this wizard? https://pytorch.org/get-started/locally/ Yes, I used that prior to commenting, and it worked. I was just pointing out an implicit requirement above the current privateGPT....

@StephenDWright I worked through a similar problem yesterday. The output of the spew when the model is loaded shows you do not have the correct LlamaCpp installed in your running...

> I am not sure why people are having troubles, I have actually ran on a clean Windows successfully, and also on several Linux machines... Clean Windows? That is the...

> @johnbrisbin Thank you for the feedback. I am also trying to run it in VS code, in a venv. I have deleted the folder and environment and cloned so...

> > 13\. Did you try the Embeddings GPU setting with the ingest process? It should make a big difference. I hope. > > It makes almost no difference. With...

How do you justify this change? It will certainly make the ingested database much smaller and reduce the RAM footprint, but the size of the chunks is also reflected in...

You must always do your installs within the virtual environment or they will not be seen when you run the code later. So, you must be within the venv when...