Rishabh Gupta comments

Repositories
Issues
Comments

Results 2 comments of


                                            Rishabh Gupta

Ollama only using half of available CPU cores with NUMA multi-socket systems

I am facing a similar situation. I have 20 CPUs but it consumes only 10. Can we tweak the configuration to improve the performance of the model ?

ERROR: Failed building wheel for llama-cpp-python

pip install llama-cpp-python --verbose Using pip 23.2.1 from /apps/workarea/PycharmProjects/llmsql/venv/lib/python3.9/site-packages/pip (python 3.9) Collecting llama-cpp-python Using cached llama_cpp_python-0.2.32.tar.gz (10.1 MB) Running command pip subprocess to install build dependencies Collecting scikit-build-core[pyproject]>=0.5.1 Obtaining dependency...