[Ollama] Error: POST predict: Post "http://127.0.0.1:36273/completion": EOF

[jaypeche]

What is the issue?

I have compiled from sources on Gentoo GNU/Linux, but when running ollama models. It returns this error :

jay@strix ~ $ ollama run llama3
>>> hello world 
Error: POST predict: Post "http://127.0.0.1:36611/completion": EOF

My system info : https://gist.github.com/jaypeche/552f5e3cada5cbd3fb918471a1b816b3

Ollama systemd logs : https://gist.github.com/jaypeche/265bd59b20d7981d88df907fa865f133

Ollama ebuild : https://github.com/gentoo/guru/blob/master/sci-ml/ollama/ollama-0.9.0.ebuild

Do you have any idea ?

Relevant log output

OS

No response

GPU

No response

CPU

No response

Ollama version

No response

[rick-github]

juin 17 13:42:38 strix ollama[43954]: CUDA error: the provided PTX was compiled with an unsupported toolchain.

Probably need to raise the issue with the maintainer of the ebuild.

[jaypeche]

I will report this issue to gentoo ebuild maintainer. Many thanks

[jaypeche]

I have unmask nvidia-drivers & nvidia-cuda-tools, to align versions of cuda compiler. And this, help me !

strix /home/jay # emerge -av nvidia-drivers nvidia-cuda-toolkit

 * IMPORTANT: 2 news items need reading for repository 'gentoo'.
 * Use eselect news read to view new items.


These are the packages that would be merged, in order:

Calculating dependencies... done!
Dependency resolution took 1.79 s (backtrack: 0/20).

[ebuild   R   ~] dev-util/nvidia-cuda-toolkit-12.9.0:0/12.9.0::gentoo  USE="-clang -debugger -examples -nsight -profiler -rdma -sanitizer" PYTHON_TARGETS="python3_12 python3_13 -python3_11" 0 KiB
[ebuild   R   ~] x11-drivers/nvidia-drivers-575.64:0/575::gentoo  USE="X modules static-libs strip tools -dist-kernel -kernel-open -modules-compress -modules-sign -persistenced -powerd -wayland" ABI_X86="32 (64)" 0 KiB

Total: 2 packages (2 reinstalls), Size of downloads: 0 KiB

Would you like to merge these packages? [Yes/No] 

Execution of ollama,with LLMs like mistral-nemo works fine !