ollama EOF of starcoder2:15b on Ollama 0.1.28

EOF of starcoder2:15b on Ollama 0.1.28

Open owenzhao opened this issue 5 months ago • 18 comments

Mac mini M1 16GB 512GB

macOS Sonoma 14.4 (23E214)

ollama run starcoder2:15b
pulling manifest 
pulling dc5deb763c38... 100% ▕████████████████████████████████████████████████▏ 9.1 GB                         
pulling 4ec42cd966c9... 100% ▕████████████████████████████████████████████████▏  12 KB                         
pulling 5671842f8d52... 100% ▕████████████████████████████████████████████████▏  346 B                         
verifying sha256 digest 
writing manifest 
removing any unused layers 
success 
Error: Post "http://127.0.0.1:11434/api/chat": EOF

ollama serve output

created by net/http.(*Transport).dialConn in goroutine 356
	net/http/transport.go:1800 +0x1060

r0      0x0
r1      0x0
r2      0x5
r3      0x1934b8848
r4      0x73
r5      0x6e
r6      0x32
r7      0x0
r8      0x600003532460
r9      0x0
r10     0x30
r11     0xc0452f67cb792c67
r12     0xc949d7c7509e6557
r13     0x386a188e1da18799
r14     0x2de4b19f0114bd9f
r15     0x4c
r16     0x1934b8900
r17     0xda000
r18     0x0
r19     0x1759c63b0
r20     0x65646f6372617473
r21     0xa
r22     0x0
r23     0x12e4e8878
r24     0x600003532460
r25     0x175003272
r26     0x1759c6428
r27     0x1759c6420
r28     0x1759c63c0
r29     0x1759c62f0
lr      0x12e3aa824
sp      0x1759c6100
pc      0x1934b8904
fault   0x0

Other models with similar size worked fine. For example, "qwen:14b" worked.

qwen:14b          	80362ced6553	8.2 GB	3 days ago

Mar 06 '24 13:03 owenzhao

I have the same issue with all three model sizes of starcoder2 in an ollama:0.1.28 Docker container running on the NVIDIA Container Toolkit.

PC: Ubuntu 22.04.4 LTS, i7-10700F, 32GB RAM, RTX 3060 12GB

Command with which the server was started:

docker run -d --rm --gpus=all -v $(realpath ~/Docker/ollama):/root/.ollama -p 11434:11434 ollama/ollama:0.1.28

Command to reproduce the error:

docker run -it --rm -e OLLAMA_HOST=10.0.10.16 ollama/ollama:0.1.28 run starcoder2:3b

If for example phi:2.7b is used instead of starcoder2:3b in the above statement, the behavior is as expected.

Mar 06 '24 16:03 KPHIBYE

The same issue with dolphincoder


ollama run dolphincoder
pulling manifest 
pulling 338f3218c250... 100% ▕████████████████████████████████████████████████████████████████████████████████████████████████████▏ 9.1 GB                         
pulling 4ec42cd966c9... 100% ▕████████████████████████████████████████████████████████████████████████████████████████████████████▏  12 KB                         
pulling 62fbfd9ed093... 100% ▕████████████████████████████████████████████████████████████████████████████████████████████████████▏  182 B                         
pulling a702f7302290... 100% ▕████████████████████████████████████████████████████████████████████████████████████████████████████▏   57 B                         
pulling d5f042e0ae3d... 100% ▕████████████████████████████████████████████████████████████████████████████████████████████████████▏  494 B                         
verifying sha256 digest 
writing manifest 
removing any unused layers 
success 
Error: Post "http://127.0.0.1:11434/api/chat": EOF

Mar 06 '24 17:03 ydaveluy

Unfortunately, I also get a similar error when I try to start the starcoder2 model with ollama.

ollama Version: 0.1.28 CPU: i7-4970K GPU: NVIDIA GeForce 1060 6GB RAM: 32 GB

I use Ollama as a Docker container in Windows WSL

time=2024-03-06T21:37:28.967Z level=INFO source=images.go:710 msg="total blobs: 26"
time=2024-03-06T21:37:28.967Z level=INFO source=images.go:717 msg="total unused blobs removed: 0"
time=2024-03-06T21:37:28.968Z level=INFO source=routes.go:1021 msg="Listening on [::]:11434 (version 0.1.28)"
time=2024-03-06T21:37:28.968Z level=INFO source=payload_common.go:107 msg="Extracting dynamic libraries..."
time=2024-03-06T21:37:31.526Z level=INFO source=payload_common.go:146 msg="Dynamic LLM libraries [rocm_v5 cuda_v11 cpu_avx2 cpu rocm_v6 cpu_avx]"
time=2024-03-06T21:37:31.526Z level=INFO source=gpu.go:94 msg="Detecting GPU type"
time=2024-03-06T21:37:31.526Z level=INFO source=gpu.go:265 msg="Searching for GPU management library libnvidia-ml.so"
time=2024-03-06T21:37:31.527Z level=INFO source=gpu.go:311 msg="Discovered GPU libraries: [/usr/lib/x86_64-linux-gnu/libnvidia-ml.so.1 /usr/lib/wsl/drivers/nv_dispig.inf_amd64_493585427225c79
4/libnvidia-ml.so.1]"
time=2024-03-06T21:37:31.534Z level=INFO source=gpu.go:99 msg="Nvidia GPU detected"
time=2024-03-06T21:37:31.534Z level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-03-06T21:37:31.540Z level=INFO source=gpu.go:146 msg="CUDA Compute Capability detected: 6.1"
time=2024-03-06T21:38:18.832Z level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-03-06T21:38:18.832Z level=INFO source=gpu.go:146 msg="CUDA Compute Capability detected: 6.1"
time=2024-03-06T21:38:18.832Z level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-03-06T21:38:18.832Z level=INFO source=gpu.go:146 msg="CUDA Compute Capability detected: 6.1"
time=2024-03-06T21:38:18.832Z level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
loading library /tmp/ollama326261946/cuda_v11/libext_server.so
time=2024-03-06T21:38:18.840Z level=INFO source=dyn_ext_server.go:90 msg="Loading Dynamic llm server: /tmp/ollama326261946/cuda_v11/libext_server.so"
time=2024-03-06T21:38:18.840Z level=INFO source=dyn_ext_server.go:150 msg="Initializing llama server"
ggml_init_cublas: GGML_CUDA_FORCE_MMQ:   yes
ggml_init_cublas: CUDA_USE_TENSOR_CORES: no
ggml_init_cublas: found 1 CUDA devices:
  Device 0: NVIDIA GeForce GTX 1060 6GB, compute capability 6.1, VMM: yes
SIGSEGV: segmentation violation
PC=0x7fef733b37fd m=14 sigcode=1
signal arrived during cgo execution

goroutine 38 [syscall]:
runtime.cgocall(0x9bd7f0, 0xc00053e6c8)
        /usr/local/go/src/runtime/cgocall.go:157 +0x4b fp=0xc00053e6a0 sp=0xc00053e668 pc=0x409b0b
github.com/jmorganca/ollama/llm._Cfunc_dyn_llama_server_init({0x7feef0001270, 0x7fef0211d350, 0x7fef0210d250, 0x7fef02110180, 0x7fef02122570, 0x7fef02117120, 0x7fef0210fe00, 0x7fef0210d2d0, 0
x7fef02122e70, 0x7fef02122110, ...}, ...)
        _cgo_gotypes.go:282 +0x45 fp=0xc00053e6c8 sp=0xc00053e6a0 pc=0x7c5c05
github.com/jmorganca/ollama/llm.newDynExtServer.func7(0xaf2e55?, 0xc?)
        /go/src/github.com/jmorganca/ollama/llm/dyn_ext_server.go:153 +0xef fp=0xc00053e7b8 sp=0xc00053e6c8 pc=0x7c714f
github.com/jmorganca/ollama/llm.newDynExtServer({0xc000720000, 0x2e}, {0xc00052a1c0, _}, {_, _, _}, {0x0, 0x0, 0x0}, ...)
        /go/src/github.com/jmorganca/ollama/llm/dyn_ext_server.go:153 +0xa65 fp=0xc00053ea58 sp=0xc00053e7b8 pc=0x7c6de5
github.com/jmorganca/ollama/llm.newLlmServer({{_, _, _}, {_, _}, {_, _}}, {_, _}, {0xc00052a1c0, ...}, ...)
        /go/src/github.com/jmorganca/ollama/llm/llm.go:158 +0x425 fp=0xc00053ec18 sp=0xc00053ea58 pc=0x7c3545
github.com/jmorganca/ollama/llm.New({0xc00036e9d8, 0x14}, {0xc00052a1c0, _}, {_, _, _}, {0x0, 0x0, 0x0}, ...)
        /go/src/github.com/jmorganca/ollama/llm/llm.go:123 +0x713 fp=0xc00053ee98 sp=0xc00053ec18 pc=0x7c2eb3
github.com/jmorganca/ollama/server.load(0xc0001f6900?, 0xc0001f6900, {{0x0, 0x800, 0x200, 0x1, 0xffffffffffffffff, 0x0, 0x0, 0x1, ...}, ...}, ...)
        /go/src/github.com/jmorganca/ollama/server/routes.go:85 +0x3a5 fp=0xc00053f018 sp=0xc00053ee98 pc=0x9971e5
github.com/jmorganca/ollama/server.ChatHandler(0xc0004feb00)
        /go/src/github.com/jmorganca/ollama/server/routes.go:1175 +0xa37 fp=0xc00053f748 sp=0xc00053f018 pc=0x9a2977
github.com/gin-gonic/gin.(*Context).Next(...)
        /root/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:174
github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func1(0xc0004feb00)
        /go/src/github.com/jmorganca/ollama/server/routes.go:945 +0x68 fp=0xc00053f780 sp=0xc00053f748 pc=0x9a11a8
github.com/gin-gonic/gin.(*Context).Next(...)
        /root/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:174
github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0xc0004feb00)
        /root/go/pkg/mod/github.com/gin-gonic/[email protected]/recovery.go:102 +0x7a fp=0xc00053f7d0 sp=0xc00053f780 pc=0x9787ba
github.com/gin-gonic/gin.(*Context).Next(...)
        /root/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:174
github.com/gin-gonic/gin.LoggerWithConfig.func1(0xc0004feb00)
        /root/go/pkg/mod/github.com/gin-gonic/[email protected]/logger.go:240 +0xde fp=0xc00053f980 sp=0xc00053f7d0 pc=0x97795e
github.com/gin-gonic/gin.(*Context).Next(...)
        /root/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:174
github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc00051a1a0, 0xc0004feb00)
        /root/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:620 +0x65b fp=0xc00053fb08 sp=0xc00053f980 pc=0x976a1b
github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc00051a1a0, {0x1179fa40?, 0xc0001fc700}, 0xc0004fea00)
        /root/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:576 +0x1dd fp=0xc00053fb48 sp=0xc00053fb08 pc=0x9761dd
net/http.serverHandler.ServeHTTP({0x1179dd60?}, {0x1179fa40?, 0xc0001fc700?}, 0x6?)
        /usr/local/go/src/net/http/server.go:2938 +0x8e fp=0xc00053fb78 sp=0xc00053fb48 pc=0x6ced4e
net/http.(*conn).serve(0xc0001f41b0, {0x117a10a8, 0xc000524000})
        /usr/local/go/src/net/http/server.go:2009 +0x5f4 fp=0xc00053ffb8 sp=0xc00053fb78 pc=0x6cac34
net/http.(*Server).Serve.func3()
        /usr/local/go/src/net/http/server.go:3086 +0x28 fp=0xc00053ffe0 sp=0xc00053ffb8 pc=0x6cf568
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00053ffe8 sp=0xc00053ffe0 pc=0x46e2c1
created by net/http.(*Server).Serve in goroutine 1
        /usr/local/go/src/net/http/server.go:3086 +0x5cb

goroutine 1 [IO wait]:
runtime.gopark(0x480f10?, 0xc0001db850?, 0xa0?, 0xb8?, 0x4f711d?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc0001db830 sp=0xc0001db810 pc=0x43e7ee
runtime.netpollblock(0x46c332?, 0x4092a6?, 0x0?)
        /usr/local/go/src/runtime/netpoll.go:564 +0xf7 fp=0xc0001db868 sp=0xc0001db830 pc=0x437277
internal/poll.runtime_pollWait(0x7fef2a5e1e28, 0x72)
        /usr/local/go/src/runtime/netpoll.go:343 +0x85 fp=0xc0001db888 sp=0xc0001db868 pc=0x468a05
internal/poll.(*pollDesc).wait(0xc000500080?, 0x4?, 0x0)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0001db8b0 sp=0xc0001db888 pc=0x4efd67
internal/poll.(*pollDesc).waitRead(...)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc000500080)
        /usr/local/go/src/internal/poll/fd_unix.go:611 +0x2ac fp=0xc0001db958 sp=0xc0001db8b0 pc=0x4f524c
net.(*netFD).accept(0xc000500080)
        /usr/local/go/src/net/fd_unix.go:172 +0x29 fp=0xc0001dba10 sp=0xc0001db958 pc=0x56be29
net.(*TCPListener).accept(0xc0004d5540)
        /usr/local/go/src/net/tcpsock_posix.go:152 +0x1e fp=0xc0001dba38 sp=0xc0001dba10 pc=0x580c3e
net.(*TCPListener).Accept(0xc0004d5540)
        /usr/local/go/src/net/tcpsock.go:315 +0x30 fp=0xc0001dba68 sp=0xc0001dba38 pc=0x57fdf0
net/http.(*onceCloseListener).Accept(0xc0001f41b0?)
        <autogenerated>:1 +0x24 fp=0xc0001dba80 sp=0xc0001dba68 pc=0x6f1ae4
net/http.(*Server).Serve(0xc000412ff0, {0x1179f830, 0xc0004d5540})
        /usr/local/go/src/net/http/server.go:3056 +0x364 fp=0xc0001dbbb0 sp=0xc0001dba80 pc=0x6cf1a4
github.com/jmorganca/ollama/server.Serve({0x1179f830, 0xc0004d5540})
        /go/src/github.com/jmorganca/ollama/server/routes.go:1048 +0x454 fp=0xc0001dbc98 sp=0xc0001dbbb0 pc=0x9a1654
github.com/jmorganca/ollama/cmd.RunServer(0xc0004fe300?, {0x11be88c0?, 0x4?, 0xadab0a?})
        /go/src/github.com/jmorganca/ollama/cmd/cmd.go:706 +0x1b9 fp=0xc0001dbd30 sp=0xc0001dbc98 pc=0x9b4799
github.com/spf13/cobra.(*Command).execute(0xc0004b9500, {0x11be88c0, 0x0, 0x0})
        /root/go/pkg/mod/github.com/spf13/[email protected]/command.go:940 +0x87c fp=0xc0001dbe68 sp=0xc0001dbd30 pc=0x764d9c
github.com/spf13/cobra.(*Command).ExecuteC(0xc0004b8900)
        /root/go/pkg/mod/github.com/spf13/[email protected]/command.go:1068 +0x3a5 fp=0xc0001dbf20 sp=0xc0001dbe68 pc=0x7655c5
github.com/spf13/cobra.(*Command).Execute(...)
        /root/go/pkg/mod/github.com/spf13/[email protected]/command.go:992
github.com/spf13/cobra.(*Command).ExecuteContext(...)
        /root/go/pkg/mod/github.com/spf13/[email protected]/command.go:985
main.main()
        /go/src/github.com/jmorganca/ollama/main.go:11 +0x4d fp=0xc0001dbf40 sp=0xc0001dbf20 pc=0x9bc90d
runtime.main()
        /usr/local/go/src/runtime/proc.go:267 +0x2bb fp=0xc0001dbfe0 sp=0xc0001dbf40 pc=0x43e39b
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0001dbfe8 sp=0xc0001dbfe0 pc=0x46e2c1

goroutine 2 [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000054fa8 sp=0xc000054f88 pc=0x43e7ee
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:404
runtime.forcegchelper()
        /usr/local/go/src/runtime/proc.go:322 +0xb3 fp=0xc000054fe0 sp=0xc000054fa8 pc=0x43e673
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000054fe8 sp=0xc000054fe0 pc=0x46e2c1
created by runtime.init.6 in goroutine 1
        /usr/local/go/src/runtime/proc.go:310 +0x1a

goroutine 3 [GC sweep wait]:
runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000055778 sp=0xc000055758 pc=0x43e7ee
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:404
runtime.bgsweep(0x0?)
        /usr/local/go/src/runtime/mgcsweep.go:321 +0xdf fp=0xc0000557c8 sp=0xc000055778 pc=0x42a73f
runtime.gcenable.func1()
        /usr/local/go/src/runtime/mgc.go:200 +0x25 fp=0xc0000557e0 sp=0xc0000557c8 pc=0x41f865
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000557e8 sp=0xc0000557e0 pc=0x46e2c1
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:200 +0x66

goroutine 4 [GC scavenge wait]:
runtime.gopark(0x2444025?, 0x239e4c1?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000055f70 sp=0xc000055f50 pc=0x43e7ee
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:404
runtime.(*scavengerState).park(0x11bb8c40)
        /usr/local/go/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc000055fa0 sp=0xc000055f70 pc=0x427f69
runtime.bgscavenge(0x0?)
        /usr/local/go/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc000055fc8 sp=0xc000055fa0 pc=0x428519
runtime.gcenable.func2()
        /usr/local/go/src/runtime/mgc.go:201 +0x25 fp=0xc000055fe0 sp=0xc000055fc8 pc=0x41f805
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000055fe8 sp=0xc000055fe0 pc=0x46e2c1
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:201 +0xa5

goroutine 18 [finalizer wait]:
runtime.gopark(0xad3ac0?, 0x10043f901?, 0x0?, 0x0?, 0x4469a5?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000054628 sp=0xc000054608 pc=0x43e7ee
runtime.runfinq()
        /usr/local/go/src/runtime/mfinal.go:193 +0x107 fp=0xc0000547e0 sp=0xc000054628 pc=0x41e8e7
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000547e8 sp=0xc0000547e0 pc=0x46e2c1
created by runtime.createfing in goroutine 1
        /usr/local/go/src/runtime/mfinal.go:163 +0x3d

goroutine 19 [select, locked to thread]:
runtime.gopark(0xc0000507a8?, 0x2?, 0x89?, 0xea?, 0xc0000507a4?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000050638 sp=0xc000050618 pc=0x43e7ee
runtime.selectgo(0xc0000507a8, 0xc0000507a0, 0x0?, 0x0, 0x0?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc000050758 sp=0xc000050638 pc=0x44e325
runtime.ensureSigM.func1()
        /usr/local/go/src/runtime/signal_unix.go:1014 +0x19f fp=0xc0000507e0 sp=0xc000050758 pc=0x46535f
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000507e8 sp=0xc0000507e0 pc=0x46e2c1
created by runtime.ensureSigM in goroutine 1
        /usr/local/go/src/runtime/signal_unix.go:997 +0xc8

goroutine 34 [syscall]:
runtime.notetsleepg(0x0?, 0x0?)
        /usr/local/go/src/runtime/lock_futex.go:236 +0x29 fp=0xc0001127a0 sp=0xc000112768 pc=0x411349
os/signal.signal_recv()
        /usr/local/go/src/runtime/sigqueue.go:152 +0x29 fp=0xc0001127c0 sp=0xc0001127a0 pc=0x46ac89
os/signal.loop()
        /usr/local/go/src/os/signal/signal_unix.go:23 +0x13 fp=0xc0001127e0 sp=0xc0001127c0 pc=0x6f4513
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0001127e8 sp=0xc0001127e0 pc=0x46e2c1
created by os/signal.Notify.func1.1 in goroutine 1
        /usr/local/go/src/os/signal/signal.go:151 +0x1f

goroutine 20 [chan receive]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000050f18 sp=0xc000050ef8 pc=0x43e7ee
runtime.chanrecv(0xc000223ce0, 0x0, 0x1)
        /usr/local/go/src/runtime/chan.go:583 +0x3cd fp=0xc000050f90 sp=0xc000050f18 pc=0x40beed
runtime.chanrecv1(0x0?, 0x0?)
        /usr/local/go/src/runtime/chan.go:442 +0x12 fp=0xc000050fb8 sp=0xc000050f90 pc=0x40baf2
github.com/jmorganca/ollama/server.Serve.func2()
        /go/src/github.com/jmorganca/ollama/server/routes.go:1030 +0x25 fp=0xc000050fe0 sp=0xc000050fb8 pc=0x9a16e5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000050fe8 sp=0xc000050fe0 pc=0x46e2c1
created by github.com/jmorganca/ollama/server.Serve in goroutine 1
        /go/src/github.com/jmorganca/ollama/server/routes.go:1029 +0x3c7

goroutine 39 [IO wait]:
runtime.gopark(0x0?, 0xb?, 0x0?, 0x0?, 0x8?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc0001155a0 sp=0xc000115580 pc=0x43e7ee
runtime.netpollblock(0x47f078?, 0x4092a6?, 0x0?)
        /usr/local/go/src/runtime/netpoll.go:564 +0xf7 fp=0xc0001155d8 sp=0xc0001155a0 pc=0x437277
internal/poll.runtime_pollWait(0x7fef2a5e1d30, 0x72)
        /usr/local/go/src/runtime/netpoll.go:343 +0x85 fp=0xc0001155f8 sp=0xc0001155d8 pc=0x468a05
internal/poll.(*pollDesc).wait(0xc000500580?, 0xc0005241f1?, 0x0)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc000115620 sp=0xc0001155f8 pc=0x4efd67
internal/poll.(*pollDesc).waitRead(...)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000500580, {0xc0005241f1, 0x1, 0x1})
        /usr/local/go/src/internal/poll/fd_unix.go:164 +0x27a fp=0xc0001156b8 sp=0xc000115620 pc=0x4f105a
net.(*netFD).Read(0xc000500580, {0xc0005241f1?, 0x0?, 0x0?})
        /usr/local/go/src/net/fd_posix.go:55 +0x25 fp=0xc000115700 sp=0xc0001156b8 pc=0x569e05
net.(*conn).Read(0xc0005ac078, {0xc0005241f1?, 0x0?, 0x0?})
        /usr/local/go/src/net/net.go:179 +0x45 fp=0xc000115748 sp=0xc000115700 pc=0x5780a5
net.(*TCPConn).Read(0x0?, {0xc0005241f1?, 0x0?, 0x0?})
        <autogenerated>:1 +0x25 fp=0xc000115778 sp=0xc000115748 pc=0x589fa5
net/http.(*connReader).backgroundRead(0xc0005241e0)
        /usr/local/go/src/net/http/server.go:683 +0x37 fp=0xc0001157c8 sp=0xc000115778 pc=0x6c4ab7
net/http.(*connReader).startBackgroundRead.func2()
        /usr/local/go/src/net/http/server.go:679 +0x25 fp=0xc0001157e0 sp=0xc0001157c8 pc=0x6c49e5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0001157e8 sp=0xc0001157e0 pc=0x46e2c1
created by net/http.(*connReader).startBackgroundRead in goroutine 38
        /usr/local/go/src/net/http/server.go:679 +0xba

goroutine 35 [GC worker (idle)]:
runtime.gopark(0x2d31695dec?, 0x3?, 0xb8?, 0x48?, 0xc000052fd0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000052f50 sp=0xc000052f30 pc=0x43e7ee
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000052fe0 sp=0xc000052f50 pc=0x4213e5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000052fe8 sp=0xc000052fe0 pc=0x46e2c1
created by runtime.gcBgMarkStartWorkers in goroutine 24
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 5 [GC worker (idle)]:
runtime.gopark(0x2d31695dec?, 0x3?, 0x16?, 0xdf?, 0xc0000517d0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000051750 sp=0xc000051730 pc=0x43e7ee
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0000517e0 sp=0xc000051750 pc=0x4213e5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000517e8 sp=0xc0000517e0 pc=0x46e2c1
created by runtime.gcBgMarkStartWorkers in goroutine 24
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 6 [GC worker (idle)]:
runtime.gopark(0x2d31695b94?, 0x3?, 0xfb?, 0xc1?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000056750 sp=0xc000056730 pc=0x43e7ee
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0000567e0 sp=0xc000056750 pc=0x4213e5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000567e8 sp=0xc0000567e0 pc=0x46e2c1
created by runtime.gcBgMarkStartWorkers in goroutine 24
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 7 [GC worker (idle)]:
runtime.gopark(0x2d31695b30?, 0x3?, 0xad?, 0xee?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000056f50 sp=0xc000056f30 pc=0x43e7ee
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000056fe0 sp=0xc000056f50 pc=0x4213e5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000056fe8 sp=0xc000056fe0 pc=0x46e2c1
created by runtime.gcBgMarkStartWorkers in goroutine 24
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 36 [GC worker (idle)]:
runtime.gopark(0x2d2b789b83?, 0x1?, 0x38?, 0x31?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000112f50 sp=0xc000112f30 pc=0x43e7ee
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000112fe0 sp=0xc000112f50 pc=0x4213e5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000112fe8 sp=0xc000112fe0 pc=0x46e2c1
created by runtime.gcBgMarkStartWorkers in goroutine 24
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 50 [GC worker (idle)]:
runtime.gopark(0x2d31695d88?, 0x3?, 0x70?, 0x30?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000608750 sp=0xc000608730 pc=0x43e7ee
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0006087e0 sp=0xc000608750 pc=0x4213e5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0006087e8 sp=0xc0006087e0 pc=0x46e2c1
created by runtime.gcBgMarkStartWorkers in goroutine 24
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 8 [GC worker (idle)]:
runtime.gopark(0x2d31695c5c?, 0x1?, 0x5e?, 0x5?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000057750 sp=0xc000057730 pc=0x43e7ee
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0000577e0 sp=0xc000057750 pc=0x4213e5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000577e8 sp=0xc0000577e0 pc=0x46e2c1
created by runtime.gcBgMarkStartWorkers in goroutine 24
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 37 [GC worker (idle)]:
runtime.gopark(0x2d3169b760?, 0x3?, 0x10?, 0xe?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000113750 sp=0xc000113730 pc=0x43e7ee
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0001137e0 sp=0xc000113750 pc=0x4213e5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0001137e8 sp=0xc0001137e0 pc=0x46e2c1
created by runtime.gcBgMarkStartWorkers in goroutine 24
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

rax    0x0
rbx    0x7fef089fe410
rcx    0x67
rdx    0x0
rdi    0x0
rsi    0x0
rbp    0x7fef089fe3d0
rsp    0x7fef089fe238
r8     0x7feeb737a8d0
r9     0x7feeb737a8f8
r10    0x7fef73225b40
r11    0x7fef733afa80
r12    0x0
r13    0x7feef0cacae8
r14    0x7feef0202550
r15    0x0
rip    0x7fef733b37fd
rflags 0x10283
cs     0x33
fs     0x0
gs     0x0

Mar 06 '24 21:03 schueszy

Same here: Windows 11 23H2 Windows Docker WSL Crashes with the same error when loading dolphincoder and starcoder2 any flavor.

Mar 06 '24 23:03 dmerriman11

Arch ollama-cuda package 0.1.28, getting EOF on 3b, 7b, 15b:q4, 15b

Mar 06 19:08:34 beast ollama[51611]:   Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes
Mar 06 19:08:34 beast ollama[51611]: SIGSEGV: segmentation violation
Mar 06 19:08:34 beast ollama[51611]: PC=0x7913ca217fdd m=10 sigcode=1 addr=0x0
Mar 06 19:08:34 beast ollama[51611]: signal arrived during cgo execution
Mar 06 19:08:34 beast ollama[51611]: goroutine 33 gp=0xc0005001c0 m=10 mp=0xc000600808 [syscall]:
Mar 06 19:08:34 beast ollama[51611]: runtime.cgocall(0x607ed1f3e8e0, 0xc000032730)
Mar 06 19:08:34 beast ollama[51611]:         runtime/cgocall.go:157 +0x4b fp=0xc000032708 sp=0xc0000326d0 pc=0x607ed19703eb
Mar 06 19:08:34 beast ollama[51611]: github.com/jmorganca/ollama/llm._Cfunc_dyn_llama_server_init({0x79134c0023c0, 0x791348d231e0, 0x791348d1e380, 0x791348d1f4f0, 0x791348d24c60, 0x791348d28460, 0x791348d25950, 0x
791348d24e20, 0x791348d25b30, 0x791348d25f10, ...}, ...)
Mar 06 19:08:34 beast ollama[51611]:         _cgo_gotypes.go:295 +0x47 fp=0xc000032730 sp=0xc000032708 pc=0x607ed1d48587
Mar 06 19:08:34 beast ollama[51611]: github.com/jmorganca/ollama/llm.newDynExtServer.func7(0xc0005400f0, 0xc0004f7e60)
Mar 06 19:08:34 beast ollama[51611]:         github.com/jmorganca/ollama/llm/dyn_ext_server.go:153 +0x112 fp=0xc000032870 sp=0xc000032730 pc=0x607ed1d49c52
Mar 06 19:08:34 beast ollama[51611]: github.com/jmorganca/ollama/llm.newDynExtServer({0xc00037f200, 0x2f}, {0xc0001c0310, _}, {_, _, _}, {0x0, 0x0, 0x0}, ...)
Mar 06 19:08:34 beast ollama[51611]:         github.com/jmorganca/ollama/llm/dyn_ext_server.go:153 +0xac5 fp=0xc000032ab8 sp=0xc000032870 pc=0x607ed1d49885
Mar 06 19:08:34 beast ollama[51611]: github.com/jmorganca/ollama/llm.newLlmServer({{_, _, _}, {_, _}, {_, _}}, {_, _}, {0xc0001c0310, ...}, ...)
Mar 06 19:08:34 beast ollama[51611]:         github.com/jmorganca/ollama/llm/llm.go:158 +0x4c5 fp=0xc000032c78 sp=0xc000032ab8 pc=0x607ed1d45ca5
Mar 06 19:08:34 beast ollama[51611]: github.com/jmorganca/ollama/llm.New({0xc000302588, 0x15}, {0xc0001c0310, _}, {_, _, _}, {0x0, 0x0, 0x0}, ...)

Mar 07 '24 01:03 jonstelly

Same SIGSEGV: segmentation violation when using starcoder2 here: RTX4090 ollama/ollama:0.1.28 on ubuntu 23.10

Mar 07 '24 09:03 tbund

Same for me on MacBook M3Max with starcoder2 and dolphincoder models. Others work fine.

Mar 07 '24 11:03 miharekar

Same for me with ROCm on an 7900 XTX. starcoder2:latest and starcoder2:15b fail and kill the server. codellama:latest and mixtral:latest work fine.

Mar 07 '24 15:03 vrinek

Getting an error with 0.1.28 Error: Post "http://0.0.0.0:11434/api/chat": EOF

Mar 08 '24 05:03 samavedulark

Also crashing for raspberry pi 5 running in docker, for both starcoder2 and dolphincoder. other models run fine as well

SIGSEGV: segmentation violation

Mar 08 '24 05:03 reikolydia

To provide a datapoint; it seems to work fine on my Intel Macbook (32GB i9)

Mar 08 '24 19:03 jimscard

The problem seems to be solved https://github.com/ollama/ollama/issues/3011#issuecomment-1986453562 It now works for me after pulling the latest Docker image

Mar 08 '24 22:03 KPHIBYE

Thanks for the update, resolving this issue for now. The problem was that the docker image was not updated. Anyone seeing this should be able to get it running by pulling the new docker image. Let me know if that is not the case.

Mar 08 '24 22:03 BruceMacD

For people who use it without a container but rather as a command-line utility, it’s not fixed at all. And brew reinstall ollama doesn’t help. So, it’s not fixed entirely yet. Does the Homebrew package need an update as well?

Mar 08 '24 23:03 Aeron

Thanks for the update, resolving this issue for now. The problem was that the docker image was not updated. Anyone seeing this should be able to get it running by pulling the new docker image. Let me know if that is not the case.

No. I am using Ollama from brew in Mac, not a docker.

Mar 09 '24 01:03 owenzhao

Gonna chime in, like in https://github.com/ollama/ollama/issues/2899#issuecomment-1983444744 but updated to now;

Setup: docker 24.0.5, RTX 3090, AMD Ryzen 7 3700X, NixOS 23.11.4976.79baff8812a0;

Seems like whatever update has been made to ollama/ollama:0.1.27 in the last half a day period has managed to get at least starcoder2:3b-q5_K_M to not SIGSEVS, but instead repeat the prompt infinitely. starcoder2:7b-q5_K_M is not emitting anything in ~30mins for the same trivial prompt without segfault.

In case any of that helps.

Mar 09 '24 01:03 d-goldin

I just updated by brew upgrade. However, the issue is still happening. I use the homebrew on MacAire M1, without docker.

Mar 09 '24 02:03 Tigerfyj

Yup, still an issue for me - I use it with brew. Tried reinstalling go, cmake, and ollama, and no diff.

It does indeed work fine in docker container, but I want to run it natively 😅

In case it helps, my full stack trace:

time=2024-03-09T08:21:44.219+01:00 level=INFO source=routes.go:1021 msg="Listening on 127.0.0.1:11434 (version 0.1.28)"
time=2024-03-09T08:21:44.219+01:00 level=INFO source=payload_common.go:107 msg="Extracting dynamic libraries..."
time=2024-03-09T08:21:44.236+01:00 level=INFO source=payload_common.go:146 msg="Dynamic LLM libraries [metal]"
[GIN] 2024/03/09 - 08:21:51 | 200 |      68.666µs |       127.0.0.1 | HEAD     "/"
[GIN] 2024/03/09 - 08:21:51 | 200 |   11.077625ms |       127.0.0.1 | DELETE   "/api/delete"
[GIN] 2024/03/09 - 08:21:54 | 200 |      35.125µs |       127.0.0.1 | HEAD     "/"
[GIN] 2024/03/09 - 08:21:54 | 404 |     308.625µs |       127.0.0.1 | POST     "/api/show"
time=2024-03-09T08:21:55.767+01:00 level=INFO source=download.go:136 msg="downloading 28bfdfaeba9f in 18 100 MB part(s)"
time=2024-03-09T08:22:30.568+01:00 level=INFO source=download.go:136 msg="downloading 4ec42cd966c9 in 1 12 KB part(s)"
time=2024-03-09T08:22:33.382+01:00 level=INFO source=download.go:136 msg="downloading 9211f2fea414 in 1 345 B part(s)"
[GIN] 2024/03/09 - 08:22:37 | 200 | 43.220304958s |       127.0.0.1 | POST     "/api/pull"
[GIN] 2024/03/09 - 08:22:37 | 200 |     513.291µs |       127.0.0.1 | POST     "/api/show"
[GIN] 2024/03/09 - 08:22:37 | 200 |     523.208µs |       127.0.0.1 | POST     "/api/show"
time=2024-03-09T08:22:37.520+01:00 level=INFO source=dyn_ext_server.go:90 msg="Loading Dynamic llm server: /var/folders/ll/3b7rx3rd50536l2sgvnl5lqc0000gn/T/ollama4252121922/metal/libext_server.dylib"
time=2024-03-09T08:22:37.520+01:00 level=INFO source=dyn_ext_server.go:150 msg="Initializing llama server"
SIGSEGV: segmentation violation
PC=0x197224904 m=4 sigcode=2 addr=0x0
signal arrived during cgo execution

goroutine 82 gp=0x14000684380 m=4 mp=0x14000098008 [syscall]:
runtime.cgocall(0x1028dae20, 0x14000030728)
	runtime/cgocall.go:157 +0x44 fp=0x140000306f0 sp=0x140000306b0 pc=0x10237bd84
github.com/jmorganca/ollama/llm._Cfunc_dyn_llama_server_init({0x95b634f0, 0x14ca050c8, 0x14ca0582c, 0x14ca058fc, 0x14ca05b08, 0x14ca063b4, 0x14ca0708c, 0x14ca07078, 0x14ca07148, 0x14ca0790c, ...}, ...)
	_cgo_gotypes.go:292 +0x30 fp=0x14000030720 sp=0x140000306f0 pc=0x102706a20
github.com/jmorganca/ollama/llm.newDynExtServer.func7(0x14000182460, 0x1400062c708)
	github.com/jmorganca/ollama/llm/dyn_ext_server.go:153 +0xf0 fp=0x14000030860 sp=0x14000030720 pc=0x102707d50
github.com/jmorganca/ollama/llm.newDynExtServer({0x14000642180, 0x5b}, {0x140005381c0, 0x6d}, {0x0, 0x0, _}, {_, _, _}, ...)
	github.com/jmorganca/ollama/llm/dyn_ext_server.go:153 +0x968 fp=0x14000030ab0 sp=0x14000030860 pc=0x1027079f8
github.com/jmorganca/ollama/llm.newLlmServer({{0x0, 0x0, 0x0}, {_, _}, {_, _}}, {_, _}, {_, ...}, ...)
	github.com/jmorganca/ollama/llm/llm.go:158 +0x3d0 fp=0x14000030c70 sp=0x14000030ab0 pc=0x1027041b0
github.com/jmorganca/ollama/llm.New({0x140001b3770, 0x41}, {0x140005381c0, 0x6d}, {0x0, 0x0, _}, {_, _, _}, ...)
	github.com/jmorganca/ollama/llm/llm.go:123 +0x43c fp=0x14000030eb0 sp=0x14000030c70 pc=0x102703c2c
github.com/jmorganca/ollama/server.load(0x1400024c200, 0x14000156180, {{0x0, 0x800, 0x200, 0x1, 0xffffffffffffffff, 0x0, 0x0, 0x1, ...}, ...}, ...)
	github.com/jmorganca/ollama/server/routes.go:85 +0x330 fp=0x14000031020 sp=0x14000030eb0 pc=0x1028b4b60
github.com/jmorganca/ollama/server.ChatHandler(0x1400024c200)
	github.com/jmorganca/ollama/server/routes.go:1175 +0x8fc fp=0x14000031730 sp=0x14000031020 pc=0x1028be8dc
github.com/gin-gonic/gin.(*Context).Next(...)
	github.com/gin-gonic/[email protected]/context.go:174
github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func1(0x1400024c200)
	github.com/jmorganca/ollama/server/routes.go:945 +0x78 fp=0x14000031770 sp=0x14000031730 pc=0x1028bd2c8
github.com/gin-gonic/gin.(*Context).Next(...)
	github.com/gin-gonic/[email protected]/context.go:174
github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0x1400024c200)
	github.com/gin-gonic/[email protected]/recovery.go:102 +0x80 fp=0x140000317c0 sp=0x14000031770 pc=0x102899050
github.com/gin-gonic/gin.(*Context).Next(...)
	github.com/gin-gonic/[email protected]/context.go:174
github.com/gin-gonic/gin.LoggerWithConfig.func1(0x1400024c200)
	github.com/gin-gonic/[email protected]/logger.go:240 +0xb0 fp=0x14000031970 sp=0x140000317c0 pc=0x1028983f0
github.com/gin-gonic/gin.(*Context).Next(...)
	github.com/gin-gonic/[email protected]/context.go:174
github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0x140004aa000, 0x1400024c200)
	github.com/gin-gonic/[email protected]/gin.go:620 +0x514 fp=0x14000031b00 sp=0x14000031970 pc=0x102897934
github.com/gin-gonic/gin.(*Engine).ServeHTTP(0x140004aa000, {0x102c58090, 0x14000632000}, 0x1400036e000)
	github.com/gin-gonic/[email protected]/gin.go:576 +0x174 fp=0x14000031b40 sp=0x14000031b00 pc=0x102897294
net/http.serverHandler.ServeHTTP({0x102c56360?}, {0x102c58090?, 0x14000632000?}, 0x6?)
	net/http/server.go:3137 +0xbc fp=0x14000031b70 sp=0x14000031b40 pc=0x102622f8c
net/http.(*conn).serve(0x140000c22d0, {0x102c59648, 0x140004a6960})
	net/http/server.go:2039 +0x508 fp=0x14000031fa0 sp=0x14000031b70 pc=0x10261e828
net/http.(*Server).Serve.gowrap3()
	net/http/server.go:3285 +0x30 fp=0x14000031fd0 sp=0x14000031fa0 pc=0x1026236d0
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x14000031fd0 sp=0x14000031fd0 pc=0x1023e6f24
created by net/http.(*Server).Serve in goroutine 1
	net/http/server.go:3285 +0x3f0

goroutine 1 gp=0x140000021c0 m=nil [IO wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x9?)
	runtime/proc.go:402 +0xc8 fp=0x140004d17c0 sp=0x140004d17a0 pc=0x1023b22a8
runtime.netpollblock(0x1400062b858?, 0x246d7c4?, 0x1?)
	runtime/netpoll.go:573 +0x158 fp=0x140004d1800 sp=0x140004d17c0 pc=0x1023abb88
internal/poll.runtime_pollWait(0x1033e3ba0, 0x72)
	runtime/netpoll.go:345 +0xa0 fp=0x140004d1830 sp=0x140004d1800 pc=0x1023e0930
internal/poll.(*pollDesc).wait(0x14000498280?, 0x10238e08c?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x28 fp=0x140004d1860 sp=0x140004d1830 pc=0x102468e28
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0x14000498280)
	internal/poll/fd_unix.go:611 +0x250 fp=0x140004d1910 sp=0x140004d1860 pc=0x10246d8b0
net.(*netFD).accept(0x14000498280)
	net/fd_unix.go:172 +0x28 fp=0x140004d19d0 sp=0x140004d1910 pc=0x1024dca08
net.(*TCPListener).accept(0x14000469760)
	net/tcpsock_posix.go:159 +0x28 fp=0x140004d1a00 sp=0x140004d19d0 pc=0x1024f0dc8
net.(*TCPListener).Accept(0x14000469760)
	net/tcpsock.go:327 +0x2c fp=0x140004d1a40 sp=0x140004d1a00 pc=0x1024f003c
net/http.(*onceCloseListener).Accept(0x140000c22d0?)
	<autogenerated>:1 +0x30 fp=0x140004d1a60 sp=0x140004d1a40 pc=0x102645410
net/http.(*Server).Serve(0x140003a4ff0, {0x102c57e20, 0x14000469760})
	net/http/server.go:3255 +0x2a8 fp=0x140004d1b90 sp=0x140004d1a60 pc=0x102623338
github.com/jmorganca/ollama/server.Serve({0x102c57e20, 0x14000469760})
	github.com/jmorganca/ollama/server/routes.go:1048 +0x3c8 fp=0x140004d1ca0 sp=0x140004d1b90 pc=0x1028bd6f8
github.com/jmorganca/ollama/cmd.RunServer(0x1400049c400?, {0x1030cf2c0?, 0x4?, 0x1028dc907?})
	github.com/jmorganca/ollama/cmd/cmd.go:706 +0x188 fp=0x140004d1d40 sp=0x140004d1ca0 pc=0x1028d3308
github.com/spf13/cobra.(*Command).execute(0x14000496f08, {0x1030cf2c0, 0x0, 0x0})
	github.com/spf13/[email protected]/command.go:940 +0x654 fp=0x140004d1e60 sp=0x140004d1d40 pc=0x1026b0144
github.com/spf13/cobra.(*Command).ExecuteC(0x14000496308)
	github.com/spf13/[email protected]/command.go:1068 +0x320 fp=0x140004d1f20 sp=0x140004d1e60 pc=0x1026b08b0
github.com/spf13/cobra.(*Command).Execute(...)
	github.com/spf13/[email protected]/command.go:992
github.com/spf13/cobra.(*Command).ExecuteContext(...)
	github.com/spf13/[email protected]/command.go:985
main.main()
	github.com/jmorganca/ollama/main.go:11 +0x54 fp=0x140004d1f40 sp=0x140004d1f20 pc=0x1028da6c4
runtime.main()
	runtime/proc.go:271 +0x28c fp=0x140004d1fd0 sp=0x140004d1f40 pc=0x1023b1e7c
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x140004d1fd0 sp=0x140004d1fd0 pc=0x1023e6f24

goroutine 2 gp=0x14000002c40 m=nil [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:402 +0xc8 fp=0x14000068f90 sp=0x14000068f70 pc=0x1023b22a8
runtime.goparkunlock(...)
	runtime/proc.go:408
runtime.forcegchelper()
	runtime/proc.go:326 +0xb8 fp=0x14000068fd0 sp=0x14000068f90 pc=0x1023b2138
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x14000068fd0 sp=0x14000068fd0 pc=0x1023e6f24
created by runtime.init.6 in goroutine 1
	runtime/proc.go:314 +0x24

goroutine 3 gp=0x14000003180 m=nil [GC sweep wait]:
runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:402 +0xc8 fp=0x14000069760 sp=0x14000069740 pc=0x1023b22a8
runtime.goparkunlock(...)
	runtime/proc.go:408
runtime.bgsweep(0x1400007a000)
	runtime/mgcsweep.go:318 +0x108 fp=0x140000697b0 sp=0x14000069760 pc=0x10239dfe8
runtime.gcenable.gowrap1()
	runtime/mgc.go:203 +0x28 fp=0x140000697d0 sp=0x140000697b0 pc=0x102392198
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x140000697d0 sp=0x140000697d0 pc=0x1023e6f24
created by runtime.gcenable in goroutine 1
	runtime/mgc.go:203 +0x6c

goroutine 4 gp=0x14000003340 m=nil [GC scavenge wait]:
runtime.gopark(0x5fa6a7?, 0x6553f100?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:402 +0xc8 fp=0x14000069f60 sp=0x14000069f40 pc=0x1023b22a8
runtime.goparkunlock(...)
	runtime/proc.go:408
runtime.(*scavengerState).park(0x103067680)
	runtime/mgcscavenge.go:425 +0x5c fp=0x14000069f90 sp=0x14000069f60 pc=0x10239b96c
runtime.bgscavenge(0x1400007a000)
	runtime/mgcscavenge.go:658 +0xac fp=0x14000069fb0 sp=0x14000069f90 pc=0x10239bf2c
runtime.gcenable.gowrap2()
	runtime/mgc.go:204 +0x28 fp=0x14000069fd0 sp=0x14000069fb0 pc=0x102392138
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x14000069fd0 sp=0x14000069fd0 pc=0x1023e6f24
created by runtime.gcenable in goroutine 1
	runtime/mgc.go:204 +0xac

goroutine 18 gp=0x1400009c700 m=nil [finalizer wait]:
runtime.gopark(0x0?, 0x140000baf90?, 0xe0?, 0x1?, 0x1000000010?)
	runtime/proc.go:402 +0xc8 fp=0x14000068580 sp=0x14000068560 pc=0x1023b22a8
runtime.runfinq()
	runtime/mfinal.go:194 +0x108 fp=0x140000687d0 sp=0x14000068580 pc=0x102391268
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x140000687d0 sp=0x140000687d0 pc=0x1023e6f24
created by runtime.createfing in goroutine 1
	runtime/mfinal.go:164 +0x80

goroutine 19 gp=0x1400009d880 m=nil [select, locked to thread]:
runtime.gopark(0x140000647a0?, 0x2?, 0x0?, 0x0?, 0x14000064790?)
	runtime/proc.go:402 +0xc8 fp=0x14000064640 sp=0x14000064620 pc=0x1023b22a8
runtime.selectgo(0x140000647a0, 0x1400006478c, 0x0?, 0x0, 0x0?, 0x1)
	runtime/select.go:327 +0x608 fp=0x14000064750 sp=0x14000064640 pc=0x1023c4c98
runtime.ensureSigM.func1()
	runtime/signal_unix.go:1034 +0x168 fp=0x140000647d0 sp=0x14000064750 pc=0x1023dd448
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x140000647d0 sp=0x140000647d0 pc=0x1023e6f24
created by runtime.ensureSigM in goroutine 1
	runtime/signal_unix.go:1017 +0xd8

goroutine 20 gp=0x1400009da40 m=5 mp=0x1400006fb08 [syscall]:
runtime.sigNoteSleep(0x0?)
	runtime/os_darwin.go:132 +0x20 fp=0x14000064f90 sp=0x14000064f50 pc=0x1023acc50
os/signal.signal_recv()
	runtime/sigqueue.go:149 +0x2c fp=0x14000064fb0 sp=0x14000064f90 pc=0x1023e304c
os/signal.loop()
	os/signal/signal_unix.go:23 +0x1c fp=0x14000064fd0 sp=0x14000064fb0 pc=0x102647b2c
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x14000064fd0 sp=0x14000064fd0 pc=0x1023e6f24
created by os/signal.Notify.func1.1 in goroutine 1
	os/signal/signal.go:151 +0x28

goroutine 34 gp=0x14000504380 m=nil [chan receive]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:402 +0xc8 fp=0x140005166f0 sp=0x140005166d0 pc=0x1023b22a8
runtime.chanrecv(0x140001c1680, 0x0, 0x1)
	runtime/chan.go:583 +0x404 fp=0x14000516770 sp=0x140005166f0 pc=0x10237e274
runtime.chanrecv1(0x0?, 0x0?)
	runtime/chan.go:442 +0x14 fp=0x140005167a0 sp=0x14000516770 pc=0x10237de34
github.com/jmorganca/ollama/server.Serve.func2()
	github.com/jmorganca/ollama/server/routes.go:1030 +0x2c fp=0x140005167d0 sp=0x140005167a0 pc=0x1028bd78c
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x140005167d0 sp=0x140005167d0 pc=0x1023e6f24
created by github.com/jmorganca/ollama/server.Serve in goroutine 1
	github.com/jmorganca/ollama/server/routes.go:1029 +0x368

goroutine 35 gp=0x14000504540 m=nil [GC worker (idle)]:
runtime.gopark(0x1f861bc40188?, 0x3?, 0xf?, 0xac?, 0x0?)
	runtime/proc.go:402 +0xc8 fp=0x14000516f30 sp=0x14000516f10 pc=0x1023b22a8
runtime.gcBgMarkWorker()
	runtime/mgc.go:1310 +0xd8 fp=0x14000516fd0 sp=0x14000516f30 pc=0x102394288
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x14000516fd0 sp=0x14000516fd0 pc=0x1023e6f24
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1234 +0x28

goroutine 5 gp=0x14000003880 m=nil [GC worker (idle)]:
runtime.gopark(0x1f861bc3fdf3?, 0x3?, 0x59?, 0x7c?, 0x0?)
	runtime/proc.go:402 +0xc8 fp=0x1400006a730 sp=0x1400006a710 pc=0x1023b22a8
runtime.gcBgMarkWorker()
	runtime/mgc.go:1310 +0xd8 fp=0x1400006a7d0 sp=0x1400006a730 pc=0x102394288
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x1400006a7d0 sp=0x1400006a7d0 pc=0x1023e6f24
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1234 +0x28

goroutine 36 gp=0x14000504700 m=nil [GC worker (idle)]:
runtime.gopark(0x1f861bc3ff6a?, 0x1?, 0x20?, 0xc4?, 0x0?)
	runtime/proc.go:402 +0xc8 fp=0x14000517730 sp=0x14000517710 pc=0x1023b22a8
runtime.gcBgMarkWorker()
	runtime/mgc.go:1310 +0xd8 fp=0x140005177d0 sp=0x14000517730 pc=0x102394288
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x140005177d0 sp=0x140005177d0 pc=0x1023e6f24
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1234 +0x28

goroutine 50 gp=0x14000584380 m=nil [GC worker (idle)]:
runtime.gopark(0x1f861bc405ed?, 0x1?, 0x28?, 0x70?, 0x0?)
	runtime/proc.go:402 +0xc8 fp=0x14000512730 sp=0x14000512710 pc=0x1023b22a8
runtime.gcBgMarkWorker()
	runtime/mgc.go:1310 +0xd8 fp=0x140005127d0 sp=0x14000512730 pc=0x102394288
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x140005127d0 sp=0x140005127d0 pc=0x1023e6f24
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1234 +0x28

goroutine 66 gp=0x14000602000 m=nil [GC worker (idle)]:
runtime.gopark(0x1f861bc4c5d2?, 0x3?, 0xb8?, 0x64?, 0x0?)
	runtime/proc.go:402 +0xc8 fp=0x14000608730 sp=0x14000608710 pc=0x1023b22a8
runtime.gcBgMarkWorker()
	runtime/mgc.go:1310 +0xd8 fp=0x140006087d0 sp=0x14000608730 pc=0x102394288
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x140006087d0 sp=0x140006087d0 pc=0x1023e6f24
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1234 +0x28

goroutine 37 gp=0x140005048c0 m=nil [GC worker (idle)]:
runtime.gopark(0x1f861bc3ee00?, 0x1?, 0xfb?, 0xed?, 0x0?)
	runtime/proc.go:402 +0xc8 fp=0x14000517f30 sp=0x14000517f10 pc=0x1023b22a8
runtime.gcBgMarkWorker()
	runtime/mgc.go:1310 +0xd8 fp=0x14000517fd0 sp=0x14000517f30 pc=0x102394288
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x14000517fd0 sp=0x14000517fd0 pc=0x1023e6f24
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1234 +0x28

goroutine 38 gp=0x14000504a80 m=nil [GC worker (idle)]:
runtime.gopark(0x1f861afbf80f?, 0x14000590040?, 0x1a?, 0xa?, 0x0?)
	runtime/proc.go:402 +0xc8 fp=0x14000518730 sp=0x14000518710 pc=0x1023b22a8
runtime.gcBgMarkWorker()
	runtime/mgc.go:1310 +0xd8 fp=0x140005187d0 sp=0x14000518730 pc=0x102394288
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x140005187d0 sp=0x140005187d0 pc=0x1023e6f24
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1234 +0x28

goroutine 67 gp=0x140006021c0 m=nil [GC worker (idle)]:
runtime.gopark(0x1f861bc40c6f?, 0x1?, 0x6b?, 0x67?, 0x0?)
	runtime/proc.go:402 +0xc8 fp=0x14000608f30 sp=0x14000608f10 pc=0x1023b22a8
runtime.gcBgMarkWorker()
	runtime/mgc.go:1310 +0xd8 fp=0x14000608fd0 sp=0x14000608f30 pc=0x102394288
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x14000608fd0 sp=0x14000608fd0 pc=0x1023e6f24
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1234 +0x28

goroutine 39 gp=0x14000504c40 m=nil [GC worker (idle)]:
runtime.gopark(0x1f861bc4037c?, 0x1?, 0x1c?, 0x89?, 0x0?)
	runtime/proc.go:402 +0xc8 fp=0x14000518f30 sp=0x14000518f10 pc=0x1023b22a8
runtime.gcBgMarkWorker()
	runtime/mgc.go:1310 +0xd8 fp=0x14000518fd0 sp=0x14000518f30 pc=0x102394288
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x14000518fd0 sp=0x14000518fd0 pc=0x1023e6f24
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1234 +0x28

goroutine 40 gp=0x14000504e00 m=nil [GC worker (idle)]:
runtime.gopark(0x1f861bc40616?, 0x1?, 0x82?, 0xfb?, 0x0?)
	runtime/proc.go:402 +0xc8 fp=0x14000519730 sp=0x14000519710 pc=0x1023b22a8
runtime.gcBgMarkWorker()
	runtime/mgc.go:1310 +0xd8 fp=0x140005197d0 sp=0x14000519730 pc=0x102394288
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x140005197d0 sp=0x140005197d0 pc=0x1023e6f24
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1234 +0x28

goroutine 68 gp=0x14000602380 m=nil [GC worker (idle)]:
runtime.gopark(0x1f861bc4216e?, 0x3?, 0x66?, 0xd8?, 0x0?)
	runtime/proc.go:402 +0xc8 fp=0x14000609730 sp=0x14000609710 pc=0x1023b22a8
runtime.gcBgMarkWorker()
	runtime/mgc.go:1310 +0xd8 fp=0x140006097d0 sp=0x14000609730 pc=0x102394288
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x140006097d0 sp=0x140006097d0 pc=0x1023e6f24
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1234 +0x28

goroutine 69 gp=0x14000602540 m=nil [GC worker (idle)]:
runtime.gopark(0x1f861bc4015e?, 0x3?, 0x82?, 0xe5?, 0x0?)
	runtime/proc.go:402 +0xc8 fp=0x14000609f30 sp=0x14000609f10 pc=0x1023b22a8
runtime.gcBgMarkWorker()
	runtime/mgc.go:1310 +0xd8 fp=0x14000609fd0 sp=0x14000609f30 pc=0x102394288
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x14000609fd0 sp=0x14000609fd0 pc=0x1023e6f24
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1234 +0x28

goroutine 70 gp=0x14000602700 m=nil [GC worker (idle)]:
runtime.gopark(0x1f861bc402d5?, 0x3?, 0xfa?, 0x76?, 0x0?)
	runtime/proc.go:402 +0xc8 fp=0x1400060a730 sp=0x1400060a710 pc=0x1023b22a8
runtime.gcBgMarkWorker()
	runtime/mgc.go:1310 +0xd8 fp=0x1400060a7d0 sp=0x1400060a730 pc=0x102394288
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x1400060a7d0 sp=0x1400060a7d0 pc=0x1023e6f24
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1234 +0x28

goroutine 71 gp=0x140006028c0 m=nil [GC worker (idle)]:
runtime.gopark(0x1f861bc3ffe7?, 0x3?, 0xd1?, 0xbb?, 0x0?)
	runtime/proc.go:402 +0xc8 fp=0x1400060af30 sp=0x1400060af10 pc=0x1023b22a8
runtime.gcBgMarkWorker()
	runtime/mgc.go:1310 +0xd8 fp=0x1400060afd0 sp=0x1400060af30 pc=0x102394288
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x1400060afd0 sp=0x1400060afd0 pc=0x1023e6f24
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1234 +0x28

goroutine 41 gp=0x14000504fc0 m=nil [GC worker (idle)]:
runtime.gopark(0x1030d1320?, 0x1?, 0x2e?, 0x27?, 0x0?)
	runtime/proc.go:402 +0xc8 fp=0x14000519f30 sp=0x14000519f10 pc=0x1023b22a8
runtime.gcBgMarkWorker()
	runtime/mgc.go:1310 +0xd8 fp=0x14000519fd0 sp=0x14000519f30 pc=0x102394288
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x14000519fd0 sp=0x14000519fd0 pc=0x1023e6f24
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1234 +0x28

goroutine 72 gp=0x14000602a80 m=nil [GC worker (idle)]:
runtime.gopark(0x1f861bc4c501?, 0x1?, 0xc7?, 0x41?, 0x0?)
	runtime/proc.go:402 +0xc8 fp=0x1400060b730 sp=0x1400060b710 pc=0x1023b22a8
runtime.gcBgMarkWorker()
	runtime/mgc.go:1310 +0xd8 fp=0x1400060b7d0 sp=0x1400060b730 pc=0x102394288
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x1400060b7d0 sp=0x1400060b7d0 pc=0x1023e6f24
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1234 +0x28

goroutine 87 gp=0x14000505180 m=nil [IO wait]:
runtime.gopark(0xffffffffffffffff?, 0xffffffffffffffff?, 0x23?, 0x0?, 0x1023fc600?)
	runtime/proc.go:402 +0xc8 fp=0x140004bd650 sp=0x140004bd630 pc=0x1023b22a8
runtime.netpollblock(0x0?, 0x0?, 0x0?)
	runtime/netpoll.go:573 +0x158 fp=0x140004bd690 sp=0x140004bd650 pc=0x1023abb88
internal/poll.runtime_pollWait(0x1033e39b0, 0x72)
	runtime/netpoll.go:345 +0xa0 fp=0x140004bd6c0 sp=0x140004bd690 pc=0x1023e0930
internal/poll.(*pollDesc).wait(0x14000036280?, 0x1400048b500?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x28 fp=0x140004bd6f0 sp=0x140004bd6c0 pc=0x102468e28
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x14000036280, {0x1400048b500, 0x1500, 0x1500})
	internal/poll/fd_unix.go:164 +0x200 fp=0x140004bd790 sp=0x140004bd6f0 pc=0x10246a170
net.(*netFD).Read(0x14000036280, {0x1400048b500?, 0x14bf4a308?, 0x140008cc930?})
	net/fd_posix.go:55 +0x28 fp=0x140004bd7e0 sp=0x140004bd790 pc=0x1024dae28
net.(*conn).Read(0x1400060e028, {0x1400048b500?, 0x140004bd8d8?, 0x1023856bc?})
	net/net.go:179 +0x34 fp=0x140004bd830 sp=0x140004bd7e0 pc=0x1024e82f4
net.(*TCPConn).Read(0x140004bd8a8?, {0x1400048b500?, 0x14000468300?, 0x140008e4000?})
	<autogenerated>:1 +0x2c fp=0x140004bd860 sp=0x140004bd830 pc=0x1024f8ecc
crypto/tls.(*atLeastReader).Read(0x140008cc930, {0x1400048b500?, 0x0?, 0x140008cc930?})
	crypto/tls/conn.go:806 +0x40 fp=0x140004bd8b0 sp=0x140004bd860 pc=0x10256ba90
bytes.(*Buffer).ReadFrom(0x140001e09b0, {0x102c553c0, 0x140008cc930})
	bytes/buffer.go:211 +0x90 fp=0x140004bd910 sp=0x140004bd8b0 pc=0x1024200e0
crypto/tls.(*Conn).readFromUntil(0x140001e0708, {0x1034b8d38, 0x1400060e028}, 0x140004bd9b0?)
	crypto/tls/conn.go:828 +0xd0 fp=0x140004bd950 sp=0x140004bd910 pc=0x10256bc70
crypto/tls.(*Conn).readRecordOrCCS(0x140001e0708, 0x0)
	crypto/tls/conn.go:626 +0x35c fp=0x140004bdbd0 sp=0x140004bd950 pc=0x10256940c
crypto/tls.(*Conn).readRecord(...)
	crypto/tls/conn.go:588
crypto/tls.(*Conn).Read(0x140001e0708, {0x140006c1000, 0x1000, 0x1023e28ac?})
	crypto/tls/conn.go:1370 +0x168 fp=0x140004bdc40 sp=0x140004bdbd0 pc=0x10256f398
bufio.(*Reader).Read(0x140006af320, {0x14000442660, 0x9, 0x140001b0a1c?})
	bufio/bufio.go:241 +0x1b4 fp=0x140004bdc80 sp=0x140004bdc40 pc=0x1025a46a4
io.ReadAtLeast({0x102c54840, 0x140006af320}, {0x14000442660, 0x9, 0x9}, 0x9)
	io/io.go:335 +0xa0 fp=0x140004bdcd0 sp=0x140004bdc80 pc=0x102416c80
io.ReadFull(...)
	io/io.go:354
net/http.http2readFrameHeader({0x14000442660, 0x9, 0x140000f5300?}, {0x102c54840?, 0x140006af320?})
	net/http/h2_bundle.go:1638 +0x58 fp=0x140004bdd20 sp=0x140004bdcd0 pc=0x1025df3f8
net/http.(*http2Framer).ReadFrame(0x14000442620)
	net/http/h2_bundle.go:1902 +0x78 fp=0x140004bddd0 sp=0x140004bdd20 pc=0x1025dfae8
net/http.(*http2clientConnReadLoop).run(0x140004bdf98)
	net/http/h2_bundle.go:9303 +0xf8 fp=0x140004bdf40 sp=0x140004bddd0 pc=0x102600fe8
net/http.(*http2ClientConn).readLoop(0x140001b0900)
	net/http/h2_bundle.go:9198 +0x5c fp=0x140004bdfb0 sp=0x140004bdf40 pc=0x10260050c
net/http.(*http2Transport).newClientConn.gowrap1()
	net/http/h2_bundle.go:7848 +0x28 fp=0x140004bdfd0 sp=0x140004bdfb0 pc=0x1025f9478
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x140004bdfd0 sp=0x140004bdfd0 pc=0x1023e6f24
created by net/http.(*http2Transport).newClientConn in goroutine 86
	net/http/h2_bundle.go:7848 +0xa84

goroutine 10 gp=0x14000584540 m=nil [IO wait]:
runtime.gopark(0xffffffffffffffff?, 0xffffffffffffffff?, 0x23?, 0x0?, 0x1023fc600?)
	runtime/proc.go:402 +0xc8 fp=0x14000080650 sp=0x14000080630 pc=0x1023b22a8
runtime.netpollblock(0x0?, 0x0?, 0x0?)
	runtime/netpoll.go:573 +0x158 fp=0x14000080690 sp=0x14000080650 pc=0x1023abb88
internal/poll.runtime_pollWait(0x1033e38b8, 0x72)
	runtime/netpoll.go:345 +0xa0 fp=0x140000806c0 sp=0x14000080690 pc=0x1023e0930
internal/poll.(*pollDesc).wait(0x14000036300?, 0x1400023d500?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x28 fp=0x140000806f0 sp=0x140000806c0 pc=0x102468e28
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x14000036300, {0x1400023d500, 0x1500, 0x1500})
	internal/poll/fd_unix.go:164 +0x200 fp=0x14000080790 sp=0x140000806f0 pc=0x10246a170
net.(*netFD).Read(0x14000036300, {0x1400023d500?, 0x1030d1320?, 0x1400062c030?})
	net/fd_posix.go:55 +0x28 fp=0x140000807e0 sp=0x14000080790 pc=0x1024dae28
net.(*conn).Read(0x14000790058, {0x1400023d500?, 0x140000808d8?, 0x1023856bc?})
	net/net.go:179 +0x34 fp=0x14000080830 sp=0x140000807e0 pc=0x1024e82f4
net.(*TCPConn).Read(0x140000808a8?, {0x1400023d500?, 0x1400063c300?, 0x1400027e000?})
	<autogenerated>:1 +0x2c fp=0x14000080860 sp=0x14000080830 pc=0x1024f8ecc
crypto/tls.(*atLeastReader).Read(0x1400062c030, {0x1400023d500?, 0x0?, 0x1400062c030?})
	crypto/tls/conn.go:806 +0x40 fp=0x140000808b0 sp=0x14000080860 pc=0x10256ba90
bytes.(*Buffer).ReadFrom(0x140003e4630, {0x102c553c0, 0x1400062c030})
	bytes/buffer.go:211 +0x90 fp=0x14000080910 sp=0x140000808b0 pc=0x1024200e0
crypto/tls.(*Conn).readFromUntil(0x140003e4388, {0x1034b8d38, 0x14000790058}, 0x140000809b0?)
	crypto/tls/conn.go:828 +0xd0 fp=0x14000080950 sp=0x14000080910 pc=0x10256bc70
crypto/tls.(*Conn).readRecordOrCCS(0x140003e4388, 0x0)
	crypto/tls/conn.go:626 +0x35c fp=0x14000080bd0 sp=0x14000080950 pc=0x10256940c
crypto/tls.(*Conn).readRecord(...)
	crypto/tls/conn.go:588
crypto/tls.(*Conn).Read(0x140003e4388, {0x14000159000, 0x1000, 0x1023e28ac?})
	crypto/tls/conn.go:1370 +0x168 fp=0x14000080c40 sp=0x14000080bd0 pc=0x10256f398
bufio.(*Reader).Read(0x14000152600, {0x140001b4580, 0x9, 0x1400015611c?})
	bufio/bufio.go:241 +0x1b4 fp=0x14000080c80 sp=0x14000080c40 pc=0x1025a46a4
io.ReadAtLeast({0x102c54840, 0x14000152600}, {0x140001b4580, 0x9, 0x9}, 0x9)
	io/io.go:335 +0xa0 fp=0x14000080cd0 sp=0x14000080c80 pc=0x102416c80
io.ReadFull(...)
	io/io.go:354
net/http.http2readFrameHeader({0x140001b4580, 0x9, 0x140002a0480?}, {0x102c54840?, 0x14000152600?})
	net/http/h2_bundle.go:1638 +0x58 fp=0x14000080d20 sp=0x14000080cd0 pc=0x1025df3f8
net/http.(*http2Framer).ReadFrame(0x140001b4540)
	net/http/h2_bundle.go:1902 +0x78 fp=0x14000080dd0 sp=0x14000080d20 pc=0x1025dfae8
net/http.(*http2clientConnReadLoop).run(0x14000080f98)
	net/http/h2_bundle.go:9303 +0xf8 fp=0x14000080f40 sp=0x14000080dd0 pc=0x102600fe8
net/http.(*http2ClientConn).readLoop(0x14000156000)
	net/http/h2_bundle.go:9198 +0x5c fp=0x14000080fb0 sp=0x14000080f40 pc=0x10260050c
net/http.(*http2Transport).newClientConn.gowrap1()
	net/http/h2_bundle.go:7848 +0x28 fp=0x14000080fd0 sp=0x14000080fb0 pc=0x1025f9478
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x14000080fd0 sp=0x14000080fd0 pc=0x1023e6f24
created by net/http.(*http2Transport).newClientConn in goroutine 9
	net/http/h2_bundle.go:7848 +0xa84

goroutine 91 gp=0x14000684c40 m=nil [IO wait]:
runtime.gopark(0xffffffffffffffff?, 0xffffffffffffffff?, 0x23?, 0x0?, 0x1023fc600?)
	runtime/proc.go:402 +0xc8 fp=0x14000694670 sp=0x14000694650 pc=0x1023b22a8
runtime.netpollblock(0x0?, 0x0?, 0x0?)
	runtime/netpoll.go:573 +0x158 fp=0x140006946b0 sp=0x14000694670 pc=0x1023abb88
internal/poll.runtime_pollWait(0x1033e37c0, 0x72)
	runtime/netpoll.go:345 +0xa0 fp=0x140006946e0 sp=0x140006946b0 pc=0x1023e0930
internal/poll.(*pollDesc).wait(0x140001ba000?, 0x14000740000?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x28 fp=0x14000694710 sp=0x140006946e0 pc=0x102468e28
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x140001ba000, {0x14000740000, 0xa000, 0xa000})
	internal/poll/fd_unix.go:164 +0x200 fp=0x140006947b0 sp=0x14000694710 pc=0x10246a170
net.(*netFD).Read(0x140001ba000, {0x14000740000?, 0x14bf4a308?, 0x140008cc990?})
	net/fd_posix.go:55 +0x28 fp=0x14000694800 sp=0x140006947b0 pc=0x1024dae28
net.(*conn).Read(0x1400060e088, {0x14000740000?, 0x140006948f8?, 0x1023856bc?})
	net/net.go:179 +0x34 fp=0x14000694850 sp=0x14000694800 pc=0x1024e82f4
net.(*TCPConn).Read(0x140006948c8?, {0x14000740000?, 0x140006948e8?, 0x1025a8c34?})
	<autogenerated>:1 +0x2c fp=0x14000694880 sp=0x14000694850 pc=0x1024f8ecc
crypto/tls.(*atLeastReader).Read(0x140008cc990, {0x14000740000?, 0x0?, 0x140008cc990?})
	crypto/tls/conn.go:806 +0x40 fp=0x140006948d0 sp=0x14000694880 pc=0x10256ba90
bytes.(*Buffer).ReadFrom(0x140001e0d30, {0x102c553c0, 0x140008cc990})
	bytes/buffer.go:211 +0x90 fp=0x14000694930 sp=0x140006948d0 pc=0x1024200e0
crypto/tls.(*Conn).readFromUntil(0x140001e0a88, {0x1034b8d38, 0x1400060e088}, 0x140006949d0?)
	crypto/tls/conn.go:828 +0xd0 fp=0x14000694970 sp=0x14000694930 pc=0x10256bc70
crypto/tls.(*Conn).readRecordOrCCS(0x140001e0a88, 0x0)
	crypto/tls/conn.go:626 +0x35c fp=0x14000694bf0 sp=0x14000694970 pc=0x10256940c
crypto/tls.(*Conn).readRecord(...)
	crypto/tls/conn.go:588
crypto/tls.(*Conn).Read(0x140001e0a88, {0x140006d3000, 0x1000, 0x10237d108?})
	crypto/tls/conn.go:1370 +0x168 fp=0x14000694c60 sp=0x14000694bf0 pc=0x10256f398
net/http.(*persistConn).Read(0x140001dc240, {0x140006d3000?, 0x10237d6c0?, 0x140000a9260?})
	net/http/transport.go:1977 +0x50 fp=0x14000694cc0 sp=0x14000694c60 pc=0x1026364f0
bufio.(*Reader).fill(0x140006d1860)
	bufio/bufio.go:110 +0xf8 fp=0x14000694d00 sp=0x14000694cc0 pc=0x1025a40c8
bufio.(*Reader).Peek(0x140006d1860, 0x1)
	bufio/bufio.go:148 +0x60 fp=0x14000694d20 sp=0x14000694d00 pc=0x1025a4230
net/http.(*persistConn).readLoop(0x140001dc240)
	net/http/transport.go:2141 +0x158 fp=0x14000694fb0 sp=0x14000694d20 pc=0x102637478
net/http.(*Transport).dialConn.gowrap2()
	net/http/transport.go:1799 +0x28 fp=0x14000694fd0 sp=0x14000694fb0 pc=0x102635a88
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x14000694fd0 sp=0x14000694fd0 pc=0x1023e6f24
created by net/http.(*Transport).dialConn in goroutine 43
	net/http/transport.go:1799 +0x1018

goroutine 92 gp=0x14000003dc0 m=nil [select]:
runtime.gopark(0x14000695f38?, 0x2?, 0xf8?, 0x5d?, 0x14000695ee4?)
	runtime/proc.go:402 +0xc8 fp=0x14000695d90 sp=0x14000695d70 pc=0x1023b22a8
runtime.selectgo(0x14000695f38, 0x14000695ee0, 0x140000f5780?, 0x0, 0x0?, 0x1)
	runtime/select.go:327 +0x608 fp=0x14000695ea0 sp=0x14000695d90 pc=0x1023c4c98
net/http.(*persistConn).writeLoop(0x140001dc240)
	net/http/transport.go:2444 +0xa0 fp=0x14000695fb0 sp=0x14000695ea0 pc=0x102638d20
net/http.(*Transport).dialConn.gowrap3()
	net/http/transport.go:1800 +0x28 fp=0x14000695fd0 sp=0x14000695fb0 pc=0x102635a28
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x14000695fd0 sp=0x14000695fd0 pc=0x1023e6f24
created by net/http.(*Transport).dialConn in goroutine 43
	net/http/transport.go:1800 +0x1060

goroutine 386 gp=0x140004f6000 m=nil [IO wait]:
runtime.gopark(0xffffffffffffffff?, 0xffffffffffffffff?, 0x23?, 0x0?, 0x1023fc600?)
	runtime/proc.go:402 +0xc8 fp=0x1400035f550 sp=0x1400035f530 pc=0x1023b22a8
runtime.netpollblock(0x0?, 0x0?, 0x0?)
	runtime/netpoll.go:573 +0x158 fp=0x1400035f590 sp=0x1400035f550 pc=0x1023abb88
internal/poll.runtime_pollWait(0x1033e3aa8, 0x72)
	runtime/netpoll.go:345 +0xa0 fp=0x1400035f5c0 sp=0x1400035f590 pc=0x1023e0930
internal/poll.(*pollDesc).wait(0x1400063e080?, 0x14000634161?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x28 fp=0x1400035f5f0 sp=0x1400035f5c0 pc=0x102468e28
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x1400063e080, {0x14000634161, 0x1, 0x1})
	internal/poll/fd_unix.go:164 +0x200 fp=0x1400035f690 sp=0x1400035f5f0 pc=0x10246a170
net.(*netFD).Read(0x1400063e080, {0x14000634161?, 0x140000a8720?, 0x0?})
	net/fd_posix.go:55 +0x28 fp=0x1400035f6e0 sp=0x1400035f690 pc=0x1024dae28
net.(*conn).Read(0x1400060e008, {0x14000634161?, 0x14000630200?, 0x0?})
	net/net.go:179 +0x34 fp=0x1400035f730 sp=0x1400035f6e0 pc=0x1024e82f4
net.(*TCPConn).Read(0x0?, {0x14000634161?, 0x1400035f7a8?, 0x1025fc16c?})
	<autogenerated>:1 +0x2c fp=0x1400035f760 sp=0x1400035f730 pc=0x1024f8ecc
net/http.(*connReader).backgroundRead(0x14000634150)
	net/http/server.go:681 +0x40 fp=0x1400035f7b0 sp=0x1400035f760 pc=0x102618e50
net/http.(*connReader).startBackgroundRead.gowrap2()
	net/http/server.go:677 +0x28 fp=0x1400035f7d0 sp=0x1400035f7b0 pc=0x102618d38
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x1400035f7d0 sp=0x1400035f7d0 pc=0x1023e6f24
created by net/http.(*connReader).startBackgroundRead in goroutine 82
	net/http/server.go:677 +0xc8

goroutine 317 gp=0x1400030e540 m=nil [IO wait]:
runtime.gopark(0xffffffffffffffff?, 0xffffffffffffffff?, 0x23?, 0x0?, 0x1023fc600?)
	runtime/proc.go:402 +0xc8 fp=0x1400016d670 sp=0x1400016d650 pc=0x1023b22a8
runtime.netpollblock(0x0?, 0x0?, 0x0?)
	runtime/netpoll.go:573 +0x158 fp=0x1400016d6b0 sp=0x1400016d670 pc=0x1023abb88
internal/poll.runtime_pollWait(0x1033e2e10, 0x72)
	runtime/netpoll.go:345 +0xa0 fp=0x1400016d6e0 sp=0x1400016d6b0 pc=0x1023e0930
internal/poll.(*pollDesc).wait(0x140001ba400?, 0x14000958000?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x28 fp=0x1400016d710 sp=0x1400016d6e0 pc=0x102468e28
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x140001ba400, {0x14000958000, 0xa000, 0xa000})
	internal/poll/fd_unix.go:164 +0x200 fp=0x1400016d7b0 sp=0x1400016d710 pc=0x10246a170
net.(*netFD).Read(0x140001ba400, {0x14000958000?, 0x14bf4b5c8?, 0x140007acbd0?})
	net/fd_posix.go:55 +0x28 fp=0x1400016d800 sp=0x1400016d7b0 pc=0x1024dae28
net.(*conn).Read(0x140007900f0, {0x14000958000?, 0x1400016d8f8?, 0x1023856bc?})
	net/net.go:179 +0x34 fp=0x1400016d850 sp=0x1400016d800 pc=0x1024e82f4
net.(*TCPConn).Read(0x1400016d8c8?, {0x14000958000?, 0x10?, 0x10?})
	<autogenerated>:1 +0x2c fp=0x1400016d880 sp=0x1400016d850 pc=0x1024f8ecc
crypto/tls.(*atLeastReader).Read(0x140007acbd0, {0x14000958000?, 0x0?, 0x140007acbd0?})
	crypto/tls/conn.go:806 +0x40 fp=0x1400016d8d0 sp=0x1400016d880 pc=0x10256ba90
bytes.(*Buffer).ReadFrom(0x140001129b0, {0x102c553c0, 0x140007acbd0})
	bytes/buffer.go:211 +0x90 fp=0x1400016d930 sp=0x1400016d8d0 pc=0x1024200e0
crypto/tls.(*Conn).readFromUntil(0x14000112708, {0x1034b8d38, 0x140007900f0}, 0x1400016d9d0?)
	crypto/tls/conn.go:828 +0xd0 fp=0x1400016d970 sp=0x1400016d930 pc=0x10256bc70
crypto/tls.(*Conn).readRecordOrCCS(0x14000112708, 0x0)
	crypto/tls/conn.go:626 +0x35c fp=0x1400016dbf0 sp=0x1400016d970 pc=0x10256940c
crypto/tls.(*Conn).readRecord(...)
	crypto/tls/conn.go:588
crypto/tls.(*Conn).Read(0x14000112708, {0x140005d7000, 0x1000, 0x10237d108?})
	crypto/tls/conn.go:1370 +0x168 fp=0x1400016dc60 sp=0x1400016dbf0 pc=0x10256f398
net/http.(*persistConn).Read(0x140001dcc60, {0x140005d7000?, 0x10237d6c0?, 0x140006c9140?})
	net/http/transport.go:1977 +0x50 fp=0x1400016dcc0 sp=0x1400016dc60 pc=0x1026364f0
bufio.(*Reader).fill(0x140006ae660)
	bufio/bufio.go:110 +0xf8 fp=0x1400016dd00 sp=0x1400016dcc0 pc=0x1025a40c8
bufio.(*Reader).Peek(0x140006ae660, 0x1)
	bufio/bufio.go:148 +0x60 fp=0x1400016dd20 sp=0x1400016dd00 pc=0x1025a4230
net/http.(*persistConn).readLoop(0x140001dcc60)
	net/http/transport.go:2141 +0x158 fp=0x1400016dfb0 sp=0x1400016dd20 pc=0x102637478
net/http.(*Transport).dialConn.gowrap2()
	net/http/transport.go:1799 +0x28 fp=0x1400016dfd0 sp=0x1400016dfb0 pc=0x102635a88
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x1400016dfd0 sp=0x1400016dfd0 pc=0x1023e6f24
created by net/http.(*Transport).dialConn in goroutine 158
	net/http/transport.go:1799 +0x1018

goroutine 318 gp=0x1400030e700 m=nil [select]:
runtime.gopark(0x140005e2f38?, 0x2?, 0xf8?, 0x2d?, 0x140005e2ee4?)
	runtime/proc.go:402 +0xc8 fp=0x140005e2d90 sp=0x140005e2d70 pc=0x1023b22a8
runtime.selectgo(0x140005e2f38, 0x140005e2ee0, 0x1400040f280?, 0x0, 0x0?, 0x1)
	runtime/select.go:327 +0x608 fp=0x140005e2ea0 sp=0x140005e2d90 pc=0x1023c4c98
net/http.(*persistConn).writeLoop(0x140001dcc60)
	net/http/transport.go:2444 +0xa0 fp=0x140005e2fb0 sp=0x140005e2ea0 pc=0x102638d20
net/http.(*Transport).dialConn.gowrap3()
	net/http/transport.go:1800 +0x28 fp=0x140005e2fd0 sp=0x140005e2fb0 pc=0x102635a28
runtime.goexit({})
	runtime/asm_arm64.s:1222 +0x4 fp=0x140005e2fd0 sp=0x140005e2fd0 pc=0x1023e6f24
created by net/http.(*Transport).dialConn in goroutine 158
	net/http/transport.go:1800 +0x1060

r0      0x0
r1      0x0
r2      0x5
r3      0x197224848
r4      0x73
r5      0x6e
r6      0x32
r7      0x0
r8      0x600000f08810
r9      0x0
r10     0x30
r11     0xc0452f67cb792c67
r12     0xc949d7c7509e6557
r13     0x386a188e1da18799
r14     0x2de4b19f0114bd9f
r15     0x1c9
r16     0x197224900
r17     0x1b800
r18     0x0
r19     0x16faae3b0
r20     0x65646f6372617473
r21     0xa
r22     0x0
r23     0x14cbcc878
r24     0x600000f08810
r25     0x16f003272
r26     0x16faae428
r27     0x16faae420
r28     0x16faae3c0
r29     0x16faae2f0
lr      0x14ca8e824
sp      0x16faae100
pc      0x197224904
fault   0x0

Mar 09 '24 07:03 miharekar

So pulling this repo and building it locally fixed it 🥳

Not sure if it's a fix from main or a building process issue, but I got it working, that's what matters.

Mar 11 '24 13:03 miharekar

So pulling this repo and building it locally fixed it 🥳

Not sure if it's a fix from main or a building process issue, but I got it working, that's what matters.

What is the version of your build Ollama? As there is a new version 0.1.29 pre released. I would like to know if it fix this issue.

Mar 11 '24 22:03 owenzhao

Hi folks, there was an original version of 0.1.28 in prerelease that didn't support Starcoder 2, and the 0.1.28 Docker image didn't receive Starcoder 2 support right away (sorry about this, it won't happen again). As of now all versions of Ollama 0.1.28 and later should support Starcoder 2.

Mar 12 '24 00:03 jmorganca

For people who use brew like me. You must uninstall ollama first then reinstall, directly reinstall will use the old version in cache.

brew uninstall ollama
brew cleanup
brew install ollama --cask

Make sure to install the cask version, as there are two ollama in brew and the other is still broken.

Mar 12 '24 01:03 owenzhao

@owenzhao nope, that still doesn't fix it for me.

~/Development/Personal/ollama on  main via  v1.22.1 took 26s
08:38:12 ❯ brew uninstall ollama
Error: Cask 'ollama' is not installed.

~/Development/Personal/ollama on  main via  v1.22.1
08:38:21 ❯ brew cleanup

~/Development/Personal/ollama on  main via  v1.22.1 took 3s
08:38:25 ❯ brew install ollama
Warning: Treating ollama as a formula. For the cask, use homebrew/cask/ollama or specify the `--cask` flag.
==> Downloading https://ghcr.io/v2/homebrew/core/ollama/manifests/0.1.28
########################################################################################################################################## 100.0%
==> Fetching ollama
==> Downloading https://ghcr.io/v2/homebrew/core/ollama/blobs/sha256:00599d0091627cc8baed988c6b6705eb2bc7b8bac366430acbf279d1a4c084e2
########################################################################################################################################## 100.0%
==> Pouring ollama--0.1.28.arm64_sonoma.bottle.tar.gz
==> Caveats
To restart ollama after an upgrade:
  brew services restart ollama
Or, if you don't want/need a background service you can just run:
  /opt/homebrew/opt/ollama/bin/ollama serve
==> Summary
🍺  /opt/homebrew/Cellar/ollama/0.1.28: 7 files, 13MB
==> Running `brew cleanup ollama`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).

~/Development/Personal/ollama on  main via  v1.22.1 took 2s
08:38:32 ❯ brew services restart ollama
Stopping `ollama`... (might take a while)
==> Successfully stopped `ollama` (label: homebrew.mxcl.ollama)
==> Successfully started `ollama` (label: homebrew.mxcl.ollama)

~/Development/Personal/ollama on  main via  v1.22.1
08:38:37 ❯ ollama run starcoder2:3b
Error: Post "http://127.0.0.1:11434/api/chat": EOF

The locally built one still works though. I have it at 41b00b9856b6708e5af165064c6ee0e556f600ab so that's quite a few commits forward from 0.1.28.

So @jmorganca not sure why you closed it. As discussed before, this isn't (only) about Docker image. I'm installing it with homebrew.

Mar 12 '24 07:03 miharekar

@owenzhao nope, that still doesn't fix it for me.

~/Development/Personal/ollama on  main via  v1.22.1 took 26s
08:38:12 ❯ brew uninstall ollama
Error: Cask 'ollama' is not installed.

~/Development/Personal/ollama on  main via  v1.22.1
08:38:21 ❯ brew cleanup

~/Development/Personal/ollama on  main via  v1.22.1 took 3s
08:38:25 ❯ brew install ollama
Warning: Treating ollama as a formula. For the cask, use homebrew/cask/ollama or specify the `--cask` flag.
==> Downloading https://ghcr.io/v2/homebrew/core/ollama/manifests/0.1.28
########################################################################################################################################## 100.0%
==> Fetching ollama
==> Downloading https://ghcr.io/v2/homebrew/core/ollama/blobs/sha256:00599d0091627cc8baed988c6b6705eb2bc7b8bac366430acbf279d1a4c084e2
########################################################################################################################################## 100.0%
==> Pouring ollama--0.1.28.arm64_sonoma.bottle.tar.gz
==> Caveats
To restart ollama after an upgrade:
  brew services restart ollama
Or, if you don't want/need a background service you can just run:
  /opt/homebrew/opt/ollama/bin/ollama serve
==> Summary
🍺  /opt/homebrew/Cellar/ollama/0.1.28: 7 files, 13MB
==> Running `brew cleanup ollama`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).

~/Development/Personal/ollama on  main via  v1.22.1 took 2s
08:38:32 ❯ brew services restart ollama
Stopping `ollama`... (might take a while)
==> Successfully stopped `ollama` (label: homebrew.mxcl.ollama)
==> Successfully started `ollama` (label: homebrew.mxcl.ollama)

~/Development/Personal/ollama on  main via  v1.22.1
08:38:37 ❯ ollama run starcoder2:3b
Error: Post "http://127.0.0.1:11434/api/chat": EOF

The locally built one still works though. I have it at 41b00b9 so that's quite a few commits forward from 0.1.28.

So @jmorganca not sure why you closed it. As discussed before, this isn't (only) about Docker image. I'm installing it with homebrew.

Sorry. There are two ollamas in brew, you must install the cask one. So the Formulae one is still broken.

brew uninstall ollama
brew install ollama --cask

Mar 12 '24 07:03 owenzhao

Right, but I don't want the app, only the binary. Shouldn't binary get fixed too?

Mar 12 '24 07:03 miharekar

Right, but I don't want the app, only the binary. Shouldn't binary get fixed too?

The cask are apps. Formulae are scripts. If you just want apps, you should always prefer cask.

Mar 12 '24 07:03 owenzhao

As I said, I don't want apps 😅

I'm using ollama from the command line only.

Mar 12 '24 07:03 miharekar

As I said, I don't want apps 😅

I'm using ollama from the command line only.

Command line are still apps. The cask is the command line app. Just like you had installed, but working.

Mar 12 '24 07:03 owenzhao

So the formulae will not get fixed?

Mar 12 '24 08:03 miharekar

As I said, I don't want apps 😅

I'm using ollama from the command line only.

OK. Let me explain more.

There are two ollamas in brew. One is called Formulae, the other is called Cask. What is the differences.

When you install something in brew, it is common that the one depends other things to work. Those are called Formulaes.

There are standalone apps that can work on itself. They are called casks.

So casks means it can work standalone. It can be a command line app or GUI app.

Mar 12 '24 08:03 owenzhao

ollama ollama copied to clipboard

EOF of starcoder2:15b on Ollama 0.1.28

ollama
ollama copied to clipboard