Error running the API server locally (MacBook Pro M1)

[alexandriabindas]

I have been running into an issue trying to run the API server locally. Any ideas on how to get past this issue?

(.venv) (base) alexbindas@Alexandrias-MBP privateGPT % python3.11 -m private_gpt 20:00:16.379 [INFO ] private_gpt.settings.settings_loader - Starting application with profiles=['default'] 20:00:17.337 [INFO ] private_gpt.components.llm.llm_component - Initializing the LLM in mode=local Traceback (most recent call last): File "/Users/alexbindas/workspace/generative-ai-project/privateGPT/.venv/lib/python3.11/site-packages/injector/init.py", line 798, in get return self._context[key] ~~~~~~~~~~~~~^^^^^ KeyError: <class 'private_gpt.ui.ui.PrivateGptUi'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/Users/alexbindas/workspace/privateGPT/.venv/lib/python3.11/site-packages/injector/init.py", line 798, in get return self._context[key] ~~~~~~~~~~~~~^^^^^ KeyError: <class 'private_gpt.server.ingest.ingest_service.IngestService'>

During handling of the above exception, another exception occurred:

...

OSError: dlopen(/Users/alexbindas/workspace/privateGPT/.venv/lib/python3.11/site-packages/llama_cpp/libllama.dylib, 0x0006): tried: '/Users/alexbindas/workspace/privateGPT/.venv/lib/python3.11/site-packages/llama_cpp/libllama.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/alexbindas/workspace/privateGPT/.venv/lib/python3.11/site-packages/llama_cpp/libllama.dylib' (no such file), '/Users/alexbindas/workspace/privateGPT/.venv/lib/python3.11/site-packages/llama_cpp/libllama.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))

[lopagela]

It might be linked to the poetry.lock.

Try the following:

1. Remove your python venv (to start from scratch)
2. Pull the latest code changes (from private GPT)
3. Delete poetry.lock
4. Restart the installation (so create a new venv, install all the dependencies)

My diagnostic is that we should get rid of the poetry.lock, as it contains packages that are specific to the one who generated it

[pharshdev]

I have tried the steps and no luck. Issue persists.

[alexandriabindas]

After rebuilding everything, and deleting the poetry.lock unfortunately it didn't work for me either, seeing the same issue

[soualid]

Same here

[github-actions[bot]]

Stale issue

[amnjdm]

This may help: CMAKE_ARGS="-DLLAMA_METAL=on -DCMAKE_APPLE_SILICON_PROCESSOR=arm64" ARCHFLAGS="-arch arm64" pip install --force-reinstall --no-cache-dir llama-cpp-python

^ I use no cache for a fresh DL of llama-cpp-python

The additional CMAKE arg: -DCMAKE_APPLE_SILICON_PROCESSOR=arm64 (which employs 'arm64' instead of 'X86_64') https://cmake.org/cmake/help/v3.23/variable/CMAKE_APPLE_SILICON_PROCESSOR.html and ARCHFLAGS="-arch arm64" ensures llama-cpp-python is built properly for Apple Silicon (M1/M2)

History

This was an issue for me with the OG (i.e. primordial) version of privateGPT and was circumvented using: ARCHFLAGS="-arch arm64" pip install --force-reinstall llama-cpp-python --no-cache-dir

Going back for a moment to installing the new version of privateGPT and the process of fixing the issue, I attempted the above to include metal support): CMAKE_ARGS="-DLLAMA_METAL=on" ARCHFLAGS="-arch arm64" pip install --force-reinstall llama-cpp-python --no-cache-dir and it would fail because CMAKE would detect X86_64:

...
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- x86 detected
...

The complication with CMAKE likely resides in the build code, however, life is quite full atm and my time is limited to assist with tracking down the issue. Hopefully this comment can help anyone with build issues on Apple Silicon (M1/M2) and/or inspire someone to track down the issue.

[CrazybutSolid]

fixed by adding

CMAKE_ARGS="-DCMAKE_OSX_ARCHITECTURES=arm64 -DCMAKE_APPLE_SILICON_PROCESSOR=arm64 -DLLAMA_METAL=on" pip install --upgrade --verbose --force-reinstall --no-cache-dir llama-cpp-python==0.2.23

Note the llama-cpp-python==0.2.23

I ran the command without it and it was updating the most recent llama-cpp-python, while poetry during installation was downgrading to 0.2.25, effectively rendering the reinstallation useless