ollama
ollama copied to clipboard
ollama
Ollama fails to start in CPU only mode
What is the issue?
Ollama fails to start properly when using in a system with only CPU mode. This happened after I upgraded to latest version i.e. 0.1.30 using the curl command as in the docs. I had to downgrade by uninstalling and installing specific version as in docs, but still the problem persists. Tried with docker after this mishap, still doesn't work.
Following are the logs using journalctl:
Apr 07 14:38:53 ollama[1344954]: time=2024-04-07T14:38:53.025Z level=INFO source=images.go:706 msg="total blobs: 0"
Apr 07 14:38:53 ollama[1344954]: time=2024-04-07T14:38:53.025Z level=INFO source=images.go:713 msg="total unused blobs removed: 0"
Apr 07 14:38:53 ollama[1344954]: time=2024-04-07T14:38:53.025Z level=INFO source=routes.go:1014 msg="Listening on 127.0.0.1:11434 (version 0.1.25)"
Apr 07 14:38:53 ollama[1344954]: time=2024-04-07T14:38:53.026Z level=INFO source=payload_common.go:107 msg="Extracting dynamic libraries..."
Apr 07 14:38:55 ollama[1344954]: time=2024-04-07T14:38:55.998Z level=INFO source=payload_common.go:146 msg="Dynamic LLM libraries [rocm_v6 cpu_avx2 cpu_avx cuda_v11 cpu rocm>
Apr 07 14:38:55 ollama[1344954]: time=2024-04-07T14:38:55.998Z level=INFO source=gpu.go:94 msg="Detecting GPU type"
Apr 07 14:38:55 ollama[1344954]: time=2024-04-07T14:38:55.998Z level=INFO source=gpu.go:262 msg="Searching for GPU management library libnvidia-ml.so"
Apr 07 14:38:56 ollama[1344954]: time=2024-04-07T14:38:56.000Z level=INFO source=gpu.go:308 msg="Discovered GPU libraries: []"
Apr 07 14:38:56 ollama[1344954]: time=2024-04-07T14:38:56.000Z level=INFO source=gpu.go:262 msg="Searching for GPU management library librocm_smi64.so"
Apr 07 14:38:56 ollama[1344954]: time=2024-04-07T14:38:56.000Z level=INFO source=gpu.go:308 msg="Discovered GPU libraries: []"
Apr 07 14:38:56 ollama[1344954]: time=2024-04-07T14:38:56.000Z level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
Apr 07 14:38:56 ollama[1344954]: time=2024-04-07T14:38:56.000Z level=INFO source=routes.go:1037 msg="no GPU detected"
All commands fail, for example:
$ ollama -v
Warning: could not connect to a running Ollama instance
Warning: client version is 0.1.25
$ ollama list
Error: something went wrong, please see the ollama server logs for details
What did you expect to see?
Expected normal functioning and launch. Updates shouldn't affect the system.
Steps to reproduce
All steps on a CPU only configuration:
Regular working version ollama - v --> 0.1.25
Update ollama curl -fsSL https://ollama.com/install.sh | sh
Check version now ollama -v --> 0.1.30 --> Doesn't start serving
Are there any recent changes that introduced the issue?
Update ollama curl -fsSL https://ollama.com/install.sh | sh
OS
Linux
Architecture
x86
Platform
No response
Ollama version
0.1.25
GPU
No response
GPU info
No response
CPU
Intel
Other software
CPU: Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz Supports:
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_goo
d nopl xtopology cpuid pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch in
vpcid_single tpr_shadow vnmi ept vpid ept_ad fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd
avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves avx512_vnni arch_capabilities
Please provide the full log of ollama serve !
This is the full log of Ollama from journalctl. It doesn't proceed after this, that's the problem.
I am facing the exact same error. The only thing we did is try to delete / create custom models from model files and the error started to appear. No additional logs about the problem is printed.
@navr32 I enabled OLLAMA_DEBUG and got some additional logs:
time=2024-04-15T06:33:25.816Z level=INFO source=images.go:804 msg="total blobs: 0"
time=2024-04-15T06:33:25.816Z level=INFO source=images.go:811 msg="total unused blobs removed: 0"
time=2024-04-15T06:33:25.816Z level=INFO source=routes.go:1118 msg="Listening on [::]:11434 (version 0.1.31)"
time=2024-04-15T06:33:25.817Z level=INFO source=payload_common.go:113 msg="Extracting dynamic libraries to /tmp/ollama727024826/runners ..."
time=2024-04-15T06:33:28.909Z level=INFO source=payload_common.go:140 msg="Dynamic LLM libraries [rocm_v60000 cpu cpu_avx2 cuda_v11 cpu_avx]"
time=2024-04-15T06:33:28.909Z level=DEBUG source=payload_common.go:141 msg="Override detection logic by setting OLLAMA_LLM_LIBRARY"
time=2024-04-15T06:33:28.909Z level=INFO source=gpu.go:115 msg="Detecting GPU type"
time=2024-04-15T06:33:28.909Z level=INFO source=gpu.go:265 msg="Searching for GPU management library libcudart.so*"
time=2024-04-15T06:33:28.909Z level=DEBUG source=gpu.go:283 msg="gpu management search paths: [/tmp/ollama727024826/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so* /home/MPCDAdmin/libcudart.so**]"
time=2024-04-15T06:33:28.910Z level=INFO source=gpu.go:311 msg="Discovered GPU libraries: [/tmp/ollama727024826/runners/cuda_v11/libcudart.so.11.0]"
wiring cudart library functions in /tmp/ollama727024826/runners/cuda_v11/libcudart.so.11.0
dlsym: cudaSetDevice
dlsym: cudaDeviceSynchronize
dlsym: cudaDeviceReset
dlsym: cudaMemGetInfo
dlsym: cudaGetDeviceCount
dlsym: cudaDeviceGetAttribute
dlsym: cudaDriverGetVersion
cudaSetDevice err: 35
time=2024-04-15T06:33:28.911Z level=INFO source=gpu.go:340 msg="Unable to load cudart CUDA management library /tmp/ollama727024826/runners/cuda_v11/libcudart.so.11.0: cudart init failure: 35"
time=2024-04-15T06:33:28.911Z level=INFO source=gpu.go:265 msg="Searching for GPU management library libnvidia-ml.so"
time=2024-04-15T06:33:28.911Z level=DEBUG source=gpu.go:283 msg="gpu management search paths: [/usr/local/cuda/lib64/libnvidia-ml.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libnvidia-ml.so* /usr/lib/x86_64-linux-gnu/libnvidia-ml.so* /usr/lib/wsl/lib/libnvidia-ml.so* /usr/lib/wsl/drivers/*/libnvidia-ml.so* /opt/cuda/lib64/libnvidia-ml.so* /usr/lib*/libnvidia-ml.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libnvidia-ml.so* /usr/lib/aarch64-linux-gnu/libnvidia-ml.so* /usr/local/lib*/libnvidia-ml.so* /opt/cuda/targets/x86_64-linux/lib/stubs/libnvidia-ml.so* /home/MPCDAdmin/libnvidia-ml.so*]"
time=2024-04-15T06:33:28.913Z level=INFO source=gpu.go:311 msg="Discovered GPU libraries: []"
time=2024-04-15T06:33:28.913Z level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-04-15T06:33:28.913Z level=DEBUG source=amd_linux.go:263 msg="amdgpu driver not detected /sys/module/amdgpu"
time=2024-04-15T06:33:28.913Z level=INFO source=routes.go:1141 msg="no GPU detected"
Was able to find a workaround for this issue. Hope it helps someone in need.
Since none of the CLI commands work in this case Eg: ollama pull phi, we use the API as an alternative to interact with ollama. So, we do POST /api/pull for {"name": "phi"}. You can now interact with ollama using the API for chat.
@Tamaya31 Were you able to solve this?
Sadly no. I went with the docker image as an alternative for now.
Are you still seeing this failure with the latest version 0.1.33? If so, can you share an updated server log? The log output you shared above ending with these 2 lines looks like a typical startup, with the server now waiting for requests from the client
Apr 07 14:38:56 ollama[1344954]: time=2024-04-07T14:38:56.000Z level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
Apr 07 14:38:56 ollama[1344954]: time=2024-04-07T14:38:56.000Z level=INFO source=routes.go:1037 msg="no GPU detected"
Hey @vishnu-dev , I think everything should be working correctly. It looks like you were connecting to it on the wrong port? If you're still seeing this, please share the server logs as @dhiltgen mentioned and we can reopen.
2024/08/18 19:11:08 routes.go:1108: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:E:\OLLAMA_MODELS OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[* http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_RUNNERS_DIR:C:\Users\Administrator\AppData\Local\Programs\Ollama\ollama_runners OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES:]" time=2024-08-18T19:11:08.699+08:00 level=INFO source=images.go:781 msg="total blobs: 7" time=2024-08-18T19:11:08.721+08:00 level=INFO source=images.go:788 msg="total unused blobs removed: 0" time=2024-08-18T19:11:08.725+08:00 level=INFO source=routes.go:1155 msg="Listening on [::]:11434 (version 0.3.3)" time=2024-08-18T19:11:08.728+08:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu_avx2 cuda_v11.3 rocm_v6.1 cpu cpu_avx]" time=2024-08-18T19:11:08.729+08:00 level=INFO source=gpu.go:205 msg="looking for compatible GPUs" time=2024-08-18T19:11:08.747+08:00 level=INFO source=gpu.go:346 msg="no compatible GPUs were discovered" time=2024-08-18T19:11:08.750+08:00 level=INFO source=types.go:105 msg="inference compute" id=0 library=cpu compute="" driver=0.0 name="" total="255.9 GiB" available="251.0 GiB"
