phi not working

[morandalex]

ollama run  phi
>>> hello
 Hello, how can I assist you today?


>>> create a js function
Error: Post "http://127.0.0.1:11434/api/generate": EOF

mistral is working on my machine. but phi not working , what is happening ?

[igorschlum]

Hi @morandalex

Can you give more info about the available memory, type of computer, version of Ollama?

It works well for me:

Last login: Mon Jan 8 18:39:10 on ttys016 (base) igor@Mac-Studio-192 ~ % ollama run phi

hello Hello! How can I assist you today?

create a js function Sure, here is an example of a simple JavaScript function that takes in two parameters (num1 and num2) and returns their sum:

function addNumbers(num1, num2) {
  return num1 + num2;
}

To use this function, you would simply call it with two numbers as arguments, like so: addNumbers(5, 7);. This will return the sum of 5 and 7, which is 12.

[morandalex]

rechecked also restarting the service mistral works phi no

uname -a
Linux achidevmsi 5.15.0-91-generic #101~20.04.1-Ubuntu SMP Thu Nov 16 14:22:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
nvidia-smi
Mon Jan  8 22:41:22 2024       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.147.05   Driver Version: 525.147.05   CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| N/A   41C    P5     5W /  35W |    790MiB /  4096MiB |      8%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      2496      G   /usr/lib/xorg/Xorg                116MiB |
|    0   N/A  N/A      3451      G   /usr/lib/xorg/Xorg                489MiB |
|    0   N/A  N/A      3632      G   xfwm4                               2MiB |
|    0   N/A  N/A     11439      G   ...cumentPictureInPictureAPI      114MiB |
|    0   N/A  N/A     31048    C+G   ...on=20240107-180120.236000       56MiB |
+-----------------------------------------------------------------------------+

EOF Error with phi

ollama rm phi
deleted 'phi'
ollama run phi
pulling manifest 
pulling 04778965089b... 100% ▕████████████████▏ 1.6 GB                         
pulling 7908abcab772... 100% ▕████████████████▏ 1.0 KB                         
pulling 774a15e6f1e5... 100% ▕████████████████▏   77 B                         
pulling 3188becd6bae... 100% ▕████████████████▏  132 B                         
pulling 0b8127ddf5ee... 100% ▕████████████████▏   42 B                         
pulling 4ce4b16d33a3... 100% ▕████████████████▏  555 B                         
verifying sha256 digest 
writing manifest 
removing any unused layers 
success 
>>> hello , 
Error: Post "http://127.0.0.1:11434/api/generate": EOF
ollama run phi
>>> hello
 Hello! How can I assist you today?


>>> can you create a js function please?
Error: Post "http://127.0.0.1:11434/api/generate": EOF

EOF error with zephyr

ollama rm zephyr
deleted 'zephyr'
ollama run zephyr
pulling manifest 
pulling 730ebed2578e... 100% ▕████████████████▏ 4.1 GB                         
pulling a406579cd136... 100% ▕████████████████▏ 1.1 KB                         
pulling a96be15318d6... 100% ▕████████████████▏  101 B                         
pulling fa956ab37b8c... 100% ▕████████████████▏   98 B                         
pulling 7d3eb0692a36... 100% ▕████████████████▏  483 B                         
verifying sha256 digest 
writing manifest 
removing any unused layers 
success 
>>> hello
Hello, how may I assist you today? Please feel free to ask any questions 
or provide instructions as needed. I'm here to help!

>>> create a js function
Error: Post "http://127.0.0.1:11434/api/generate": EOF

[morandalex]

another test with zephyr and phi

ollama run zephyr
>>> hello
Hello! How may I assist you today? Please let me know what your query is 
and I will do my best to provide an accurate response. You can ask any 
question related to a specific topic, request clarification about 
something, or just say hello as an introduction. Looking forward to 
hearing from you soon!

>>> can you help me
Error: Post "http://127.0.0.1:11434/api/generate": EOF
ollama run zephyr
>>> "can you help me?"
Of course! What specific problem or question are you facing? Please 
provide more context and details so that I can better understand your 
situation and offer appropriate assistance. You can type your message 
below or use speech-to-text functionality if you prefer to speak aloud. 
Let's work together to find a solution!

>>> "I am trying to understand why you are giving em eof"
Error: Post "http://127.0.0.1:11434/api/generate": EOF
ollama run phi
>>> can you help me?  
Error: Post "http://127.0.0.1:11434/api/generate": EOF
ollama run phi
Error: could not connect to ollama server, run 'ollama serve' to start it
sudo systemctl status ollama 
[sudo] password di ale: 
● ollama.service - Ollama Service
     Loaded: loaded (/etc/systemd/system/ollama.service; enabled; vendor preset>
     Active: active (running) since Mon 2024-01-08 23:11:10 CET; 17s ago
   Main PID: 36775 (ollama)
      Tasks: 10 (limit: 28379)
     Memory: 392.7M
     CGroup: /system.slice/ollama.service
             └─36775 /usr/local/bin/ollama serve

gen 08 23:11:10 achidevmsi systemd[1]: Started Ollama Service.
gen 08 23:11:10 achidevmsi ollama[36775]: 2024/01/08 23:11:10 images.go:834: to>
gen 08 23:11:10 achidevmsi ollama[36775]: 2024/01/08 23:11:10 images.go:841: to>
gen 08 23:11:10 achidevmsi ollama[36775]: 2024/01/08 23:11:10 routes.go:929: Li>
gen 08 23:11:10 achidevmsi ollama[36775]: 2024/01/08 23:11:10 shim_ext_server.g>
gen 08 23:11:10 achidevmsi ollama[36775]: 2024/01/08 23:11:10 gpu.go:34: Detect>
gen 08 23:11:10 achidevmsi ollama[36775]: 2024/01/08 23:11:10 gpu.go:53: Nvidia>
sudo systemctl restart ollama 
ollama run phi
>>> "can you help me?"
Error: Post "http://127.0.0.1:11434/api/generate": EOF

[igorschlum]

Hi @morandalex Can you try Dolphin Phi ? it's a 2.7B uncensored model, based on the Phi language model by Microsoft Research

ollama run dolphin-phi

You can also try another version of phi like

ollama run phi:2.7b-chat-v2-q4_1

It will help to understand your issue.

Try also to remove phi

ollama rm phi

Then reinstall phi

ollama run phi

[pdevine]

@morandalex with the Zephyr model it looks like you're running out of memory on the GPU (it looks like the GPU only has 4GB of ram), whereas it seems like Phi should work just fine. There are some improvements coming in 0.1.19 which should help w/ tight memory situations.

Can you run ollama ls | grep phi? It would be good to know what the ID (i.e. the sha256 value) for phi is, just to make certain you're using the latest version.

[morandalex]

it seems that I found the issue. I was running a machine with 48 gb of swapfile. reducing it to 16gb I solved the issue.

Seems an issue related to https://github.com/jmorganca/ollama/issues/939

[igorschlum]

@morandalex interesting. Can you close the Issue?

[jmorganca]

@morandalex sorry you hit this. Do you have the logs handy to debug? Look for CUDA error. To view the logs:

journalctl -u ollama