ollama
ollama copied to clipboard
Published
3 months ago
•
ollama
ollama
Crash upon loading any model with the ROCm GPU
Stacktrace:
llm_load_vocab: special tokens definition check successful ( 259/32000 ).
llm_load_print_meta: format = GGUF V3 (latest)
llm_load_print_meta: arch = llama
llm_load_print_meta: vocab type = SPM
llm_load_print_meta: n_vocab = 32000
llm_load_print_meta: n_merges = 0
llm_load_print_meta: n_ctx_train = 4096
llm_load_print_meta: n_embd = 4096
llm_load_print_meta: n_head = 32
llm_load_print_meta: n_head_kv = 32
llm_load_print_meta: n_layer = 40
llm_load_print_meta: n_rot = 128
llm_load_print_meta: n_embd_head_k = 128
llm_load_print_meta: n_embd_head_v = 128
llm_load_print_meta: n_gqa = 1
llm_load_print_meta: n_embd_k_gqa = 4096
llm_load_print_meta: n_embd_v_gqa = 4096
llm_load_print_meta: f_norm_eps = 0.0e+00
llm_load_print_meta: f_norm_rms_eps = 1.0e-05
llm_load_print_meta: f_clamp_kqv = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: n_ff = 11008
llm_load_print_meta: n_expert = 0
llm_load_print_meta: n_expert_used = 0
llm_load_print_meta: rope scaling = linear
llm_load_print_meta: freq_base_train = 10000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_yarn_orig_ctx = 4096
llm_load_print_meta: rope_finetuned = unknown
llm_load_print_meta: model type = 13B
llm_load_print_meta: model ftype = Q4_0
llm_load_print_meta: model params = 8.36 B
llm_load_print_meta: model size = 4.41 GiB (4.53 BPW)
llm_load_print_meta: general.name = LLaMA v2
llm_load_print_meta: BOS token = 1 '<s>'
llm_load_print_meta: EOS token = 2 '</s>'
llm_load_print_meta: UNK token = 0 '<unk>'
llm_load_print_meta: LF token = 13 '<0x0A>'
llm_load_tensors: ggml ctx size = 0.14 MiB
llm_load_tensors: using ROCm for GPU acceleration
llm_load_tensors: system memory used = 70.45 MiB
llm_load_tensors: VRAM used = 4446.30 MiB
llm_load_tensors: offloading 40 repeating layers to GPU
llm_load_tensors: offloading non-repeating layers to GPU
llm_load_tensors: offloaded 41/41 layers to GPU
...................................................................................................
llama_new_context_with_model: n_ctx = 2048
llama_new_context_with_model: freq_base = 10000.0
llama_new_context_with_model: freq_scale = 1
llama_kv_cache_init: VRAM kv self = 1280.00 MB
llama_new_context_with_model: KV self size = 1280.00 MiB, K (f16): 640.00 MiB, V (f16): 640.00 MiB
llama_build_graph: non-view tensors processed: 844/844
llama_new_context_with_model: compute buffer total size = 159.19 MiB
llama_new_context_with_model: VRAM scratch buffer: 156.00 MiB
llama_new_context_with_model: total VRAM used: 5882.31 MiB (model: 4446.30 MiB, context: 1436.00 MiB)
SIGSEGV: segmentation violation
PC=0x780302b2b380 m=18 sigcode=128
signal arrived during cgo execution
goroutine 67 [syscall]:
runtime.cgocall(0x9b3a90, 0xc000318808)
/usr/lib/go/src/runtime/cgocall.go:157 +0x4b fp=0xc0003187e0 sp=0xc0003187a8 pc=0x409b0b
github.com/jmorganca/ollama/llm._Cfunc_dyn_llama_server_init({0x78029c001620, 0x780309434970, 0x7803094350c0, 0x780309435150, 0x780309435300, 0x780309435480, 0x7803094359b0, 0x780309435990, 0x780309435a40, 0x780309435f20, ...}, ...)
_cgo_gotypes.go:284 +0x45 fp=0xc000318808 sp=0xc0003187e0 pc=0x7c25a5
github.com/jmorganca/ollama/llm.newDynExtServer.func7(0xae3c43?, 0x6c?)
/home/kainoa/Git/ollama-clean/llm/dyn_ext_server.go:142 +0xef fp=0xc0003188f8 sp=0xc000318808 pc=0x7c3a0f
github.com/jmorganca/ollama/llm.newDynExtServer({0xc000618000, 0x2e}, {0xc0001c48c0, _}, {_, _, _}, {0x0, 0x0, 0x0}, ...)
/home/kainoa/Git/ollama-clean/llm/dyn_ext_server.go:142 +0xa32 fp=0xc000318b88 sp=0xc0003188f8 pc=0x7c3752
github.com/jmorganca/ollama/llm.newLlmServer({{_, _, _}, {_, _}, {_, _}}, {_, _}, {0x0, ...}, ...)
/home/kainoa/Git/ollama-clean/llm/llm.go:147 +0x36a fp=0xc000318d48 sp=0xc000318b88 pc=0x7bff6a
github.com/jmorganca/ollama/llm.New({0x0?, 0x1000100000100?}, {0xc0001c48c0, _}, {_, _, _}, {0x0, 0x0, 0x0}, ...)
/home/kainoa/Git/ollama-clean/llm/llm.go:122 +0x6f9 fp=0xc000318fb8 sp=0xc000318d48 pc=0x7bf999
github.com/jmorganca/ollama/server.load(0xc000002f00?, 0xc000002f00, {{0x0, 0x800, 0x200, 0x1, 0xffffffffffffffff, 0x0, 0x0, 0x1, ...}, ...}, ...)
/home/kainoa/Git/ollama-clean/server/routes.go:83 +0x3a5 fp=0xc000319138 sp=0xc000318fb8 pc=0x98fde5
github.com/jmorganca/ollama/server.ChatHandler(0xc0002fc100)
/home/kainoa/Git/ollama-clean/server/routes.go:1071 +0x828 fp=0xc000319748 sp=0xc000319138 pc=0x99a728
github.com/gin-gonic/gin.(*Context).Next(...)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:174
github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func1(0xc0002fc100)
/home/kainoa/Git/ollama-clean/server/routes.go:883 +0x68 fp=0xc000319780 sp=0xc000319748 pc=0x999268
github.com/gin-gonic/gin.(*Context).Next(...)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:174
github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0xc0002fc100)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/recovery.go:102 +0x7a fp=0xc0003197d0 sp=0xc000319780 pc=0x974afa
github.com/gin-gonic/gin.(*Context).Next(...)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:174
github.com/gin-gonic/gin.LoggerWithConfig.func1(0xc0002fc100)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/logger.go:240 +0xde fp=0xc000319980 sp=0xc0003197d0 pc=0x973c9e
github.com/gin-gonic/gin.(*Context).Next(...)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:174
github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc0000e9a00, 0xc0002fc100)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:620 +0x65b fp=0xc000319b08 sp=0xc000319980 pc=0x972d5b
github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc0000e9a00, {0x1258e00?, 0xc0001c61c0}, 0xc0002fc500)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:576 +0x1dd fp=0xc000319b48 sp=0xc000319b08 pc=0x97251d
net/http.serverHandler.ServeHTTP({0x1257120?}, {0x1258e00?, 0xc0001c61c0?}, 0x6?)
/usr/lib/go/src/net/http/server.go:2938 +0x8e fp=0xc000319b78 sp=0xc000319b48 pc=0x6ce14e
net/http.(*conn).serve(0xc0001bae10, {0x125a468, 0xc0004a6720})
/usr/lib/go/src/net/http/server.go:2009 +0x5f4 fp=0xc000319fb8 sp=0xc000319b78 pc=0x6ca034
net/http.(*Server).Serve.func3()
/usr/lib/go/src/net/http/server.go:3086 +0x28 fp=0xc000319fe0 sp=0xc000319fb8 pc=0x6ce968
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000319fe8 sp=0xc000319fe0 pc=0x46e081
created by net/http.(*Server).Serve in goroutine 1
/usr/lib/go/src/net/http/server.go:3086 +0x5cb
goroutine 1 [IO wait]:
runtime.gopark(0x480890?, 0xc0003ab848?, 0x98?, 0xb8?, 0x4f687d?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc00011b828 sp=0xc00011b808 pc=0x43e60e
runtime.netpollblock(0x46c0f2?, 0x4092a6?, 0x0?)
/usr/lib/go/src/runtime/netpoll.go:564 +0xf7 fp=0xc00011b860 sp=0xc00011b828 pc=0x4370b7
internal/poll.runtime_pollWait(0x78036acc4e80, 0x72)
/usr/lib/go/src/runtime/netpoll.go:343 +0x85 fp=0xc00011b880 sp=0xc00011b860 pc=0x4688a5
internal/poll.(*pollDesc).wait(0xc000484080?, 0x4?, 0x0)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc00011b8a8 sp=0xc00011b880 pc=0x4ef4c7
internal/poll.(*pollDesc).waitRead(...)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc000484080)
/usr/lib/go/src/internal/poll/fd_unix.go:611 +0x2ac fp=0xc00011b950 sp=0xc00011b8a8 pc=0x4f49ac
net.(*netFD).accept(0xc000484080)
/usr/lib/go/src/net/fd_unix.go:172 +0x29 fp=0xc00011ba08 sp=0xc00011b950 pc=0x56b569
net.(*TCPListener).accept(0xc0004595c0)
/usr/lib/go/src/net/tcpsock_posix.go:152 +0x1e fp=0xc00011ba30 sp=0xc00011ba08 pc=0x58039e
net.(*TCPListener).Accept(0xc0004595c0)
/usr/lib/go/src/net/tcpsock.go:315 +0x30 fp=0xc00011ba60 sp=0xc00011ba30 pc=0x57f550
net/http.(*onceCloseListener).Accept(0xc0001bae10?)
<autogenerated>:1 +0x24 fp=0xc00011ba78 sp=0xc00011ba60 pc=0x6f0ee4
net/http.(*Server).Serve(0xc000396ff0, {0x1258bf0, 0xc0004595c0})
/usr/lib/go/src/net/http/server.go:3056 +0x364 fp=0xc00011bba8 sp=0xc00011ba78 pc=0x6ce5a4
github.com/jmorganca/ollama/server.Serve({0x1258bf0, 0xc0004595c0})
/home/kainoa/Git/ollama-clean/server/routes.go:970 +0x494 fp=0xc00011bc98 sp=0xc00011bba8 pc=0x999754
github.com/jmorganca/ollama/cmd.RunServer(0xc000482300?, {0x169c7a0?, 0x4?, 0xacbac1?})
/home/kainoa/Git/ollama-clean/cmd/cmd.go:690 +0x199 fp=0xc00011bd30 sp=0xc00011bc98 pc=0x9abb39
github.com/spf13/cobra.(*Command).execute(0xc000417800, {0x169c7a0, 0x0, 0x0})
/home/kainoa/go/pkg/mod/github.com/spf13/[email protected]/command.go:940 +0x87c fp=0xc00011be68 sp=0xc00011bd30 pc=0x763c9c
github.com/spf13/cobra.(*Command).ExecuteC(0xc000416c00)
/home/kainoa/go/pkg/mod/github.com/spf13/[email protected]/command.go:1068 +0x3a5 fp=0xc00011bf20 sp=0xc00011be68 pc=0x7644c5
github.com/spf13/cobra.(*Command).Execute(...)
/home/kainoa/go/pkg/mod/github.com/spf13/[email protected]/command.go:992
github.com/spf13/cobra.(*Command).ExecuteContext(...)
/home/kainoa/go/pkg/mod/github.com/spf13/[email protected]/command.go:985
main.main()
/home/kainoa/Git/ollama-clean/main.go:11 +0x4d fp=0xc00011bf40 sp=0xc00011bf20 pc=0x9b2bad
runtime.main()
/usr/lib/go/src/runtime/proc.go:267 +0x2bb fp=0xc00011bfe0 sp=0xc00011bf40 pc=0x43e1bb
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00011bfe8 sp=0xc00011bfe0 pc=0x46e081
goroutine 2 [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc000070fa8 sp=0xc000070f88 pc=0x43e60e
runtime.goparkunlock(...)
/usr/lib/go/src/runtime/proc.go:404
runtime.forcegchelper()
/usr/lib/go/src/runtime/proc.go:322 +0xb3 fp=0xc000070fe0 sp=0xc000070fa8 pc=0x43e493
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000070fe8 sp=0xc000070fe0 pc=0x46e081
created by runtime.init.6 in goroutine 1
/usr/lib/go/src/runtime/proc.go:310 +0x1a
goroutine 3 [GC sweep wait]:
runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc000071778 sp=0xc000071758 pc=0x43e60e
runtime.goparkunlock(...)
/usr/lib/go/src/runtime/proc.go:404
runtime.bgsweep(0x0?)
/usr/lib/go/src/runtime/mgcsweep.go:321 +0xdf fp=0xc0000717c8 sp=0xc000071778 pc=0x42a57f
runtime.gcenable.func1()
/usr/lib/go/src/runtime/mgc.go:200 +0x25 fp=0xc0000717e0 sp=0xc0000717c8 pc=0x41f6c5
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000717e8 sp=0xc0000717e0 pc=0x46e081
created by runtime.gcenable in goroutine 1
/usr/lib/go/src/runtime/mgc.go:200 +0x66
goroutine 4 [GC scavenge wait]:
runtime.gopark(0x104a1f?, 0xede89?, 0x0?, 0x0?, 0x0?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc000071f70 sp=0xc000071f50 pc=0x43e60e
runtime.goparkunlock(...)
/usr/lib/go/src/runtime/proc.go:404
runtime.(*scavengerState).park(0x166cb20)
/usr/lib/go/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc000071fa0 sp=0xc000071f70 pc=0x427de9
runtime.bgscavenge(0x0?)
/usr/lib/go/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc000071fc8 sp=0xc000071fa0 pc=0x428399
runtime.gcenable.func2()
/usr/lib/go/src/runtime/mgc.go:201 +0x25 fp=0xc000071fe0 sp=0xc000071fc8 pc=0x41f665
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000071fe8 sp=0xc000071fe0 pc=0x46e081
created by runtime.gcenable in goroutine 1
/usr/lib/go/src/runtime/mgc.go:201 +0xa5
goroutine 5 [finalizer wait]:
runtime.gopark(0x198?, 0xac4a80?, 0x1?, 0xf7?, 0x0?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc000070620 sp=0xc000070600 pc=0x43e60e
runtime.runfinq()
/usr/lib/go/src/runtime/mfinal.go:193 +0x107 fp=0xc0000707e0 sp=0xc000070620 pc=0x41e6e7
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000707e8 sp=0xc0000707e0 pc=0x46e081
created by runtime.createfing in goroutine 1
/usr/lib/go/src/runtime/mfinal.go:163 +0x3d
goroutine 6 [select, locked to thread]:
runtime.gopark(0xc0000727a8?, 0x2?, 0xa9?, 0xe8?, 0xc0000727a4?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc000072638 sp=0xc000072618 pc=0x43e60e
runtime.selectgo(0xc0000727a8, 0xc0000727a0, 0x0?, 0x0, 0x0?, 0x1)
/usr/lib/go/src/runtime/select.go:327 +0x725 fp=0xc000072758 sp=0xc000072638 pc=0x44e165
runtime.ensureSigM.func1()
/usr/lib/go/src/runtime/signal_unix.go:1014 +0x19f fp=0xc0000727e0 sp=0xc000072758 pc=0x46519f
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000727e8 sp=0xc0000727e0 pc=0x46e081
created by runtime.ensureSigM in goroutine 1
/usr/lib/go/src/runtime/signal_unix.go:997 +0xc8
goroutine 18 [syscall]:
runtime.notetsleepg(0x0?, 0x0?)
/usr/lib/go/src/runtime/lock_futex.go:236 +0x29 fp=0xc00006c7a0 sp=0xc00006c768 pc=0x411209
os/signal.signal_recv()
/usr/lib/go/src/runtime/sigqueue.go:152 +0x29 fp=0xc00006c7c0 sp=0xc00006c7a0 pc=0x46aa49
os/signal.loop()
/usr/lib/go/src/os/signal/signal_unix.go:23 +0x13 fp=0xc00006c7e0 sp=0xc00006c7c0 pc=0x6f3913
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00006c7e8 sp=0xc00006c7e0 pc=0x46e081
created by os/signal.Notify.func1.1 in goroutine 1
/usr/lib/go/src/os/signal/signal.go:151 +0x1f
goroutine 7 [chan receive]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc000072f18 sp=0xc000072ef8 pc=0x43e60e
runtime.chanrecv(0xc0004ac540, 0x0, 0x1)
/usr/lib/go/src/runtime/chan.go:583 +0x3cd fp=0xc000072f90 sp=0xc000072f18 pc=0x40beed
runtime.chanrecv1(0x0?, 0x0?)
/usr/lib/go/src/runtime/chan.go:442 +0x12 fp=0xc000072fb8 sp=0xc000072f90 pc=0x40baf2
github.com/jmorganca/ollama/server.Serve.func1()
/home/kainoa/Git/ollama-clean/server/routes.go:952 +0x25 fp=0xc000072fe0 sp=0xc000072fb8 pc=0x9997e5
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000072fe8 sp=0xc000072fe0 pc=0x46e081
created by github.com/jmorganca/ollama/server.Serve in goroutine 1
/home/kainoa/Git/ollama-clean/server/routes.go:951 +0x407
goroutine 62 [IO wait]:
runtime.gopark(0x75?, 0xb?, 0x0?, 0x0?, 0xa?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc00011f8f8 sp=0xc00011f8d8 pc=0x43e60e
runtime.netpollblock(0x47e9f8?, 0x4092a6?, 0x0?)
/usr/lib/go/src/runtime/netpoll.go:564 +0xf7 fp=0xc00011f930 sp=0xc00011f8f8 pc=0x4370b7
internal/poll.runtime_pollWait(0x78036acc4d88, 0x72)
/usr/lib/go/src/runtime/netpoll.go:343 +0x85 fp=0xc00011f950 sp=0xc00011f930 pc=0x4688a5
internal/poll.(*pollDesc).wait(0xc000040080?, 0xc000428000?, 0x0)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc00011f978 sp=0xc00011f950 pc=0x4ef4c7
internal/poll.(*pollDesc).waitRead(...)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000040080, {0xc000428000, 0x1000, 0x1000})
/usr/lib/go/src/internal/poll/fd_unix.go:164 +0x27a fp=0xc00011fa10 sp=0xc00011f978 pc=0x4f07ba
net.(*netFD).Read(0xc000040080, {0xc000428000?, 0x4ef985?, 0x0?})
/usr/lib/go/src/net/fd_posix.go:55 +0x25 fp=0xc00011fa58 sp=0xc00011fa10 pc=0x569545
net.(*conn).Read(0xc000074038, {0xc000428000?, 0x0?, 0xc0000b0518?})
/usr/lib/go/src/net/net.go:179 +0x45 fp=0xc00011faa0 sp=0xc00011fa58 pc=0x577805
net.(*TCPConn).Read(0xc0000b0510?, {0xc000428000?, 0x0?, 0xc00011fac0?})
<autogenerated>:1 +0x25 fp=0xc00011fad0 sp=0xc00011faa0 pc=0x589705
net/http.(*connReader).Read(0xc0000b0510, {0xc000428000, 0x1000, 0x1000})
/usr/lib/go/src/net/http/server.go:791 +0x14b fp=0xc00011fb20 sp=0xc00011fad0 pc=0x6c42eb
bufio.(*Reader).fill(0xc0004ac000)
/usr/lib/go/src/bufio/bufio.go:113 +0x103 fp=0xc00011fb58 sp=0xc00011fb20 pc=0x653ea3
bufio.(*Reader).Peek(0xc0004ac000, 0x4)
/usr/lib/go/src/bufio/bufio.go:151 +0x53 fp=0xc00011fb78 sp=0xc00011fb58 pc=0x653fd3
net/http.(*conn).serve(0xc0000fc240, {0x125a468, 0xc0004a6720})
/usr/lib/go/src/net/http/server.go:2044 +0x75c fp=0xc00011ffb8 sp=0xc00011fb78 pc=0x6ca19c
net/http.(*Server).Serve.func3()
/usr/lib/go/src/net/http/server.go:3086 +0x28 fp=0xc00011ffe0 sp=0xc00011ffb8 pc=0x6ce968
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00011ffe8 sp=0xc00011ffe0 pc=0x46e081
created by net/http.(*Server).Serve in goroutine 1
/usr/lib/go/src/net/http/server.go:3086 +0x5cb
goroutine 12 [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0xe0?, 0x2e?, 0xc0004c2fd0?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc0004c2f50 sp=0xc0004c2f30 pc=0x43e60e
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc0004c2fe0 sp=0xc0004c2f50 pc=0x421245
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0004c2fe8 sp=0xc0004c2fe0 pc=0x46e081
created by runtime.gcBgMarkStartWorkers in goroutine 11
/usr/lib/go/src/runtime/mgc.go:1219 +0x1c
goroutine 34 [GC worker (idle)]:
runtime.gopark(0xa09ea49875?, 0x3?, 0x84?, 0x3?, 0x0?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc0004be750 sp=0xc0004be730 pc=0x43e60e
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc0004be7e0 sp=0xc0004be750 pc=0x421245
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0004be7e8 sp=0xc0004be7e0 pc=0x46e081
created by runtime.gcBgMarkStartWorkers in goroutine 11
/usr/lib/go/src/runtime/mgc.go:1219 +0x1c
goroutine 13 [GC worker (idle)]:
runtime.gopark(0xa09ea48fd3?, 0x1?, 0x72?, 0x10?, 0xc0000737d0?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc000073750 sp=0xc000073730 pc=0x43e60e
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc0000737e0 sp=0xc000073750 pc=0x421245
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000737e8 sp=0xc0000737e0 pc=0x46e081
created by runtime.gcBgMarkStartWorkers in goroutine 11
/usr/lib/go/src/runtime/mgc.go:1219 +0x1c
goroutine 14 [GC worker (idle)]:
runtime.gopark(0xa09ea45121?, 0x3?, 0x96?, 0x5?, 0x0?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc0004c3750 sp=0xc0004c3730 pc=0x43e60e
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc0004c37e0 sp=0xc0004c3750 pc=0x421245
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0004c37e8 sp=0xc0004c37e0 pc=0x46e081
created by runtime.gcBgMarkStartWorkers in goroutine 11
/usr/lib/go/src/runtime/mgc.go:1219 +0x1c
goroutine 50 [GC worker (idle)]:
runtime.gopark(0xa09ea49267?, 0x1?, 0x4f?, 0xb6?, 0x0?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc000586750 sp=0xc000586730 pc=0x43e60e
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc0005867e0 sp=0xc000586750 pc=0x421245
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005867e8 sp=0xc0005867e0 pc=0x46e081
created by runtime.gcBgMarkStartWorkers in goroutine 11
/usr/lib/go/src/runtime/mgc.go:1219 +0x1c
goroutine 51 [GC worker (idle)]:
runtime.gopark(0xa09ea44f4b?, 0x1?, 0xc3?, 0xc5?, 0x0?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc000586f50 sp=0xc000586f30 pc=0x43e60e
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc000586fe0 sp=0xc000586f50 pc=0x421245
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000586fe8 sp=0xc000586fe0 pc=0x46e081
created by runtime.gcBgMarkStartWorkers in goroutine 11
/usr/lib/go/src/runtime/mgc.go:1219 +0x1c
goroutine 52 [GC worker (idle)]:
runtime.gopark(0xa09ea48ec5?, 0x1?, 0x40?, 0x34?, 0x0?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc000587750 sp=0xc000587730 pc=0x43e60e
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc0005877e0 sp=0xc000587750 pc=0x421245
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005877e8 sp=0xc0005877e0 pc=0x46e081
created by runtime.gcBgMarkStartWorkers in goroutine 11
/usr/lib/go/src/runtime/mgc.go:1219 +0x1c
goroutine 53 [GC worker (idle)]:
runtime.gopark(0xa09ea490ff?, 0x1?, 0x9e?, 0x11?, 0x0?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc000587f50 sp=0xc000587f30 pc=0x43e60e
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc000587fe0 sp=0xc000587f50 pc=0x421245
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000587fe8 sp=0xc000587fe0 pc=0x46e081
created by runtime.gcBgMarkStartWorkers in goroutine 11
/usr/lib/go/src/runtime/mgc.go:1219 +0x1c
goroutine 54 [GC worker (idle)]:
runtime.gopark(0xa09ea46909?, 0x1?, 0xb7?, 0x51?, 0x0?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc000588750 sp=0xc000588730 pc=0x43e60e
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc0005887e0 sp=0xc000588750 pc=0x421245
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005887e8 sp=0xc0005887e0 pc=0x46e081
created by runtime.gcBgMarkStartWorkers in goroutine 11
/usr/lib/go/src/runtime/mgc.go:1219 +0x1c
goroutine 55 [GC worker (idle)]:
runtime.gopark(0xa09ea450d1?, 0x3?, 0x57?, 0x4f?, 0x0?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc000588f50 sp=0xc000588f30 pc=0x43e60e
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc000588fe0 sp=0xc000588f50 pc=0x421245
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000588fe8 sp=0xc000588fe0 pc=0x46e081
created by runtime.gcBgMarkStartWorkers in goroutine 11
/usr/lib/go/src/runtime/mgc.go:1219 +0x1c
goroutine 56 [GC worker (idle)]:
runtime.gopark(0xa09ea45009?, 0x3?, 0x6a?, 0x4?, 0x0?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc000589750 sp=0xc000589730 pc=0x43e60e
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc0005897e0 sp=0xc000589750 pc=0x421245
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005897e8 sp=0xc0005897e0 pc=0x46e081
created by runtime.gcBgMarkStartWorkers in goroutine 11
/usr/lib/go/src/runtime/mgc.go:1219 +0x1c
goroutine 57 [GC worker (idle)]:
runtime.gopark(0xa09ea49177?, 0x3?, 0x6?, 0x1d?, 0x0?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc000589f50 sp=0xc000589f30 pc=0x43e60e
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc000589fe0 sp=0xc000589f50 pc=0x421245
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000589fe8 sp=0xc000589fe0 pc=0x46e081
created by runtime.gcBgMarkStartWorkers in goroutine 11
/usr/lib/go/src/runtime/mgc.go:1219 +0x1c
goroutine 58 [GC worker (idle)]:
runtime.gopark(0x169e4e0?, 0x1?, 0xaa?, 0x2d?, 0x0?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc000582750 sp=0xc000582730 pc=0x43e60e
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc0005827e0 sp=0xc000582750 pc=0x421245
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005827e8 sp=0xc0005827e0 pc=0x46e081
created by runtime.gcBgMarkStartWorkers in goroutine 11
/usr/lib/go/src/runtime/mgc.go:1219 +0x1c
goroutine 59 [GC worker (idle)]:
runtime.gopark(0xa09ea49159?, 0x3?, 0xc4?, 0x13?, 0x0?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc000582f50 sp=0xc000582f30 pc=0x43e60e
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc000582fe0 sp=0xc000582f50 pc=0x421245
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000582fe8 sp=0xc000582fe0 pc=0x46e081
created by runtime.gcBgMarkStartWorkers in goroutine 11
/usr/lib/go/src/runtime/mgc.go:1219 +0x1c
goroutine 60 [GC worker (idle)]:
runtime.gopark(0xa09ea43c3b?, 0x3?, 0xf5?, 0xc4?, 0x0?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc000583750 sp=0xc000583730 pc=0x43e60e
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc0005837e0 sp=0xc000583750 pc=0x421245
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005837e8 sp=0xc0005837e0 pc=0x46e081
created by runtime.gcBgMarkStartWorkers in goroutine 11
/usr/lib/go/src/runtime/mgc.go:1219 +0x1c
goroutine 61 [GC worker (idle)]:
runtime.gopark(0xa09ea46279?, 0xc00058a160?, 0x1a?, 0x14?, 0x0?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc000583f50 sp=0xc000583f30 pc=0x43e60e
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc000583fe0 sp=0xc000583f50 pc=0x421245
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000583fe8 sp=0xc000583fe0 pc=0x46e081
created by runtime.gcBgMarkStartWorkers in goroutine 11
/usr/lib/go/src/runtime/mgc.go:1219 +0x1c
goroutine 16 [IO wait]:
runtime.gopark(0x41e?, 0xb?, 0x0?, 0x0?, 0xc?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc0005918f8 sp=0xc0005918d8 pc=0x43e60e
runtime.netpollblock(0x47e9f8?, 0x4092a6?, 0x0?)
/usr/lib/go/src/runtime/netpoll.go:564 +0xf7 fp=0xc000591930 sp=0xc0005918f8 pc=0x4370b7
internal/poll.runtime_pollWait(0x78036acc4b98, 0x72)
/usr/lib/go/src/runtime/netpoll.go:343 +0x85 fp=0xc000591950 sp=0xc000591930 pc=0x4688a5
internal/poll.(*pollDesc).wait(0xc000436080?, 0xc000312000?, 0x0)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc000591978 sp=0xc000591950 pc=0x4ef4c7
internal/poll.(*pollDesc).waitRead(...)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000436080, {0xc000312000, 0x1000, 0x1000})
/usr/lib/go/src/internal/poll/fd_unix.go:164 +0x27a fp=0xc000591a10 sp=0xc000591978 pc=0x4f07ba
net.(*netFD).Read(0xc000436080, {0xc000312000?, 0x4ef985?, 0x0?})
/usr/lib/go/src/net/fd_posix.go:55 +0x25 fp=0xc000591a58 sp=0xc000591a10 pc=0x569545
net.(*conn).Read(0xc00025c148, {0xc000312000?, 0x0?, 0xc000395aa8?})
/usr/lib/go/src/net/net.go:179 +0x45 fp=0xc000591aa0 sp=0xc000591a58 pc=0x577805
net.(*TCPConn).Read(0xc000395aa0?, {0xc000312000?, 0x0?, 0xc00031dac0?})
<autogenerated>:1 +0x25 fp=0xc000591ad0 sp=0xc000591aa0 pc=0x589705
net/http.(*connReader).Read(0xc000395aa0, {0xc000312000, 0x1000, 0x1000})
/usr/lib/go/src/net/http/server.go:791 +0x14b fp=0xc000591b20 sp=0xc000591ad0 pc=0x6c42eb
bufio.(*Reader).fill(0xc0001a73e0)
/usr/lib/go/src/bufio/bufio.go:113 +0x103 fp=0xc000591b58 sp=0xc000591b20 pc=0x653ea3
bufio.(*Reader).Peek(0xc0001a73e0, 0x4)
/usr/lib/go/src/bufio/bufio.go:151 +0x53 fp=0xc000591b78 sp=0xc000591b58 pc=0x653fd3
net/http.(*conn).serve(0xc0001ba990, {0x125a468, 0xc0004a6720})
/usr/lib/go/src/net/http/server.go:2044 +0x75c fp=0xc000591fb8 sp=0xc000591b78 pc=0x6ca19c
net/http.(*Server).Serve.func3()
/usr/lib/go/src/net/http/server.go:3086 +0x28 fp=0xc000591fe0 sp=0xc000591fb8 pc=0x6ce968
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000591fe8 sp=0xc000591fe0 pc=0x46e081
created by net/http.(*Server).Serve in goroutine 1
/usr/lib/go/src/net/http/server.go:3086 +0x5cb
goroutine 64 [IO wait]:
runtime.gopark(0x41e?, 0xb?, 0x0?, 0x0?, 0xb?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc00058d8f8 sp=0xc00058d8d8 pc=0x43e60e
runtime.netpollblock(0x47e9f8?, 0x4092a6?, 0x0?)
/usr/lib/go/src/runtime/netpoll.go:564 +0xf7 fp=0xc00058d930 sp=0xc00058d8f8 pc=0x4370b7
internal/poll.runtime_pollWait(0x78036acc4c90, 0x72)
/usr/lib/go/src/runtime/netpoll.go:343 +0x85 fp=0xc00058d950 sp=0xc00058d930 pc=0x4688a5
internal/poll.(*pollDesc).wait(0xc000040200?, 0xc0002fa000?, 0x0)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc00058d978 sp=0xc00058d950 pc=0x4ef4c7
internal/poll.(*pollDesc).waitRead(...)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000040200, {0xc0002fa000, 0x1000, 0x1000})
/usr/lib/go/src/internal/poll/fd_unix.go:164 +0x27a fp=0xc00058da10 sp=0xc00058d978 pc=0x4f07ba
net.(*netFD).Read(0xc000040200, {0xc0002fa000?, 0x4ef985?, 0x0?})
/usr/lib/go/src/net/fd_posix.go:55 +0x25 fp=0xc00058da58 sp=0xc00058da10 pc=0x569545
net.(*conn).Read(0xc000074040, {0xc0002fa000?, 0x0?, 0xc0001d8218?})
/usr/lib/go/src/net/net.go:179 +0x45 fp=0xc00058daa0 sp=0xc00058da58 pc=0x577805
net.(*TCPConn).Read(0xc0001d8210?, {0xc0002fa000?, 0x0?, 0xc0003a7ac0?})
<autogenerated>:1 +0x25 fp=0xc00058dad0 sp=0xc00058daa0 pc=0x589705
net/http.(*connReader).Read(0xc0001d8210, {0xc0002fa000, 0x1000, 0x1000})
/usr/lib/go/src/net/http/server.go:791 +0x14b fp=0xc00058db20 sp=0xc00058dad0 pc=0x6c42eb
bufio.(*Reader).fill(0xc00009a180)
/usr/lib/go/src/bufio/bufio.go:113 +0x103 fp=0xc00058db58 sp=0xc00058db20 pc=0x653ea3
bufio.(*Reader).Peek(0xc00009a180, 0x4)
/usr/lib/go/src/bufio/bufio.go:151 +0x53 fp=0xc00058db78 sp=0xc00058db58 pc=0x653fd3
net/http.(*conn).serve(0xc0000fc3f0, {0x125a468, 0xc0004a6720})
/usr/lib/go/src/net/http/server.go:2044 +0x75c fp=0xc00058dfb8 sp=0xc00058db78 pc=0x6ca19c
net/http.(*Server).Serve.func3()
/usr/lib/go/src/net/http/server.go:3086 +0x28 fp=0xc00058dfe0 sp=0xc00058dfb8 pc=0x6ce968
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00058dfe8 sp=0xc00058dfe0 pc=0x46e081
created by net/http.(*Server).Serve in goroutine 1
/usr/lib/go/src/net/http/server.go:3086 +0x5cb
goroutine 68 [IO wait]:
runtime.gopark(0x100000000?, 0xb?, 0x0?, 0x0?, 0xd?)
/usr/lib/go/src/runtime/proc.go:398 +0xce fp=0xc00006e5a0 sp=0xc00006e580 pc=0x43e60e
runtime.netpollblock(0x47e9f8?, 0x4092a6?, 0x0?)
/usr/lib/go/src/runtime/netpoll.go:564 +0xf7 fp=0xc00006e5d8 sp=0xc00006e5a0 pc=0x4370b7
internal/poll.runtime_pollWait(0x78036acc4aa0, 0x72)
/usr/lib/go/src/runtime/netpoll.go:343 +0x85 fp=0xc00006e5f8 sp=0xc00006e5d8 pc=0x4688a5
internal/poll.(*pollDesc).wait(0xc000436180?, 0xc000438551?, 0x0)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc00006e620 sp=0xc00006e5f8 pc=0x4ef4c7
internal/poll.(*pollDesc).waitRead(...)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000436180, {0xc000438551, 0x1, 0x1})
/usr/lib/go/src/internal/poll/fd_unix.go:164 +0x27a fp=0xc00006e6b8 sp=0xc00006e620 pc=0x4f07ba
net.(*netFD).Read(0xc000436180, {0xc000438551?, 0xc00006e740?, 0x46a750?})
/usr/lib/go/src/net/fd_posix.go:55 +0x25 fp=0xc00006e700 sp=0xc00006e6b8 pc=0x569545
net.(*conn).Read(0xc00025c1f0, {0xc000438551?, 0x1?, 0xc0002ea730?})
/usr/lib/go/src/net/net.go:179 +0x45 fp=0xc00006e748 sp=0xc00006e700 pc=0x577805
net.(*TCPConn).Read(0xc000395aa0?, {0xc000438551?, 0xc0002ea730?, 0x0?})
<autogenerated>:1 +0x25 fp=0xc00006e778 sp=0xc00006e748 pc=0x589705
net/http.(*connReader).backgroundRead(0xc000438540)
/usr/lib/go/src/net/http/server.go:683 +0x37 fp=0xc00006e7c8 sp=0xc00006e778 pc=0x6c3eb7
net/http.(*connReader).startBackgroundRead.func2()
/usr/lib/go/src/net/http/server.go:679 +0x25 fp=0xc00006e7e0 sp=0xc00006e7c8 pc=0x6c3de5
runtime.goexit()
/usr/lib/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00006e7e8 sp=0xc00006e7e0 pc=0x46e081
created by net/http.(*connReader).startBackgroundRead in goroutine 67
/usr/lib/go/src/net/http/server.go:679 +0xba
rax 0x0
rbx 0x7800341b33c0
rcx 0x7802d8d00200
rdx 0x348
rdi 0x7802d8d00200
rsi 0x78003423a650
rbp 0x780310bfe910
rsp 0x780310bfe6e0
r8 0x90
r9 0x4
r10 0x3
r11 0x78029c9aa400
r12 0x17
r13 0x78029c9aa400
r14 0x78003efd1500
r15 0x78003efd16b8
rip 0x780302b2b380
rflags 0x10246
cs 0x33
fs 0x0
gs 0x0
Version: 4c54f0ddeb997cfefe4716e5631b270112975aab (built with CLBlast_DIR=/usr/lib/cmake/CLBlast ROCM_PATH=/opt/rocm go generate ./... && go build .)
facing same issue for default docker image
2024/01/22 09:49:51 images.go:810: INFO total blobs: 6
2024/01/22 09:49:51 images.go:817: INFO total unused blobs removed: 0
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.
[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
- using env: export GIN_MODE=release
- using code: gin.SetMode(gin.ReleaseMode)
[GIN-debug] POST /api/pull --> github.com/jmorganca/ollama/server.PullModelHandler (5 handlers)
[GIN-debug] POST /api/generate --> github.com/jmorganca/ollama/server.GenerateHandler (5 handlers)
[GIN-debug] POST /api/chat --> github.com/jmorganca/ollama/server.ChatHandler (5 handlers)
[GIN-debug] POST /api/embeddings --> github.com/jmorganca/ollama/server.EmbeddingHandler (5 handlers)
[GIN-debug] POST /api/create --> github.com/jmorganca/ollama/server.CreateModelHandler (5 handlers)
[GIN-debug] POST /api/push --> github.com/jmorganca/ollama/server.PushModelHandler (5 handlers)
[GIN-debug] POST /api/copy --> github.com/jmorganca/ollama/server.CopyModelHandler (5 handlers)
[GIN-debug] DELETE /api/delete --> github.com/jmorganca/ollama/server.DeleteModelHandler (5 handlers)
[GIN-debug] POST /api/show --> github.com/jmorganca/ollama/server.ShowModelHandler (5 handlers)
[GIN-debug] POST /api/blobs/:digest --> github.com/jmorganca/ollama/server.CreateBlobHandler (5 handlers)
[GIN-debug] HEAD /api/blobs/:digest --> github.com/jmorganca/ollama/server.HeadBlobHandler (5 handlers)
[GIN-debug] GET / --> github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
[GIN-debug] GET /api/tags --> github.com/jmorganca/ollama/server.ListModelsHandler (5 handlers)
[GIN-debug] GET /api/version --> github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func3 (5 handlers)
[GIN-debug] HEAD / --> github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
[GIN-debug] HEAD /api/tags --> github.com/jmorganca/ollama/server.ListModelsHandler (5 handlers)
[GIN-debug] HEAD /api/version --> github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func3 (5 handlers)
2024/01/22 09:49:51 routes.go:943: INFO Listening on [::]:11434 (version 0.0.0)
2024/01/22 09:49:51 payload_common.go:106: INFO Extracting dynamic libraries...
2024/01/22 09:49:52 payload_common.go:145: INFO Dynamic LLM libraries [rocm_v5 rocm_v6 cuda_v11 cpu cpu_avx cpu_avx2]
2024/01/22 09:49:52 gpu.go:91: INFO Detecting GPU type
2024/01/22 09:49:52 gpu.go:210: INFO Searching for GPU management library libnvidia-ml.so
2024/01/22 09:49:52 gpu.go:256: INFO Discovered GPU libraries: []
2024/01/22 09:49:52 gpu.go:210: INFO Searching for GPU management library librocm_smi64.so
2024/01/22 09:49:52 gpu.go:256: INFO Discovered GPU libraries: [/opt/rocm/lib/librocm_smi64.so.6.0.60000 /opt/rocm-6.0.0/lib/librocm_smi64.so.6.0.60000]
2024/01/22 09:49:52 gpu.go:106: INFO Radeon GPU detected
2024/01/22 09:50:03 cpu_common.go:11: INFO CPU has AVX2
2024/01/22 09:50:03 dyn_ext_server.go:90: INFO Loading Dynamic llm server: /tmp/ollama2441091586/rocm_v6/libext_server.so
2024/01/22 09:50:03 dyn_ext_server.go:139: INFO Initializing llama server
ggml_init_cublas: GGML_CUDA_FORCE_MMQ: no
ggml_init_cublas: CUDA_USE_TENSOR_CORES: yes
ggml_init_cublas: found 2 ROCm devices:
Device 0: Radeon RX 7900 XTX, compute capability 11.0, VMM: no
Device 1: AMD Radeon Graphics, compute capability 10.3, VMM: no
llama_model_loader: loaded meta data with 23 key-value pairs and 363 tensors from /root/.ollama/models/blobs/sha256:2609048d349e7c70196401be59bea7eb89a968d4642e409b0e798b34403b96c8 (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = llama
llama_model_loader: - kv 1: general.name str = LLaMA v2
llama_model_loader: - kv 2: llama.context_length u32 = 4096
llama_model_loader: - kv 3: llama.embedding_length u32 = 5120
llama_model_loader: - kv 4: llama.block_count u32 = 40
llama_model_loader: - kv 5: llama.feed_forward_length u32 = 13824
llama_model_loader: - kv 6: llama.rope.dimension_count u32 = 128
llama_model_loader: - kv 7: llama.attention.head_count u32 = 40
llama_model_loader: - kv 8: llama.attention.head_count_kv u32 = 40
llama_model_loader: - kv 9: llama.attention.layer_norm_rms_epsilon f32 = 0.000010
llama_model_loader: - kv 10: general.file_type u32 = 2
llama_model_loader: - kv 11: tokenizer.ggml.model str = llama
llama_model_loader: - kv 12: tokenizer.ggml.tokens arr[str,32000] = ["<unk>", "<s>", "</s>", "<0x00>", "<...
llama_model_loader: - kv 13: tokenizer.ggml.scores arr[f32,32000] = [0.000000, 0.000000, 0.000000, 0.0000...
llama_model_loader: - kv 14: tokenizer.ggml.token_type arr[i32,32000] = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...
llama_model_loader: - kv 15: tokenizer.ggml.merges arr[str,61249] = ["▁ t", "e r", "i n", "▁ a", "e n...
llama_model_loader: - kv 16: tokenizer.ggml.bos_token_id u32 = 1
llama_model_loader: - kv 17: tokenizer.ggml.eos_token_id u32 = 2
llama_model_loader: - kv 18: tokenizer.ggml.unknown_token_id u32 = 0
llama_model_loader: - kv 19: tokenizer.ggml.add_bos_token bool = true
llama_model_loader: - kv 20: tokenizer.ggml.add_eos_token bool = false
llama_model_loader: - kv 21: tokenizer.chat_template str = {% if messages[0]['role'] == 'system'...
llama_model_loader: - kv 22: general.quantization_version u32 = 2
llama_model_loader: - type f32: 81 tensors
llama_model_loader: - type q4_0: 281 tensors
llama_model_loader: - type q6_K: 1 tensors
llm_load_vocab: special tokens definition check successful ( 259/32000 ).
llm_load_print_meta: format = GGUF V3 (latest)
llm_load_print_meta: arch = llama
llm_load_print_meta: vocab type = SPM
llm_load_print_meta: n_vocab = 32000
llm_load_print_meta: n_merges = 0
llm_load_print_meta: n_ctx_train = 4096
llm_load_print_meta: n_embd = 5120
llm_load_print_meta: n_head = 40
llm_load_print_meta: n_head_kv = 40
llm_load_print_meta: n_layer = 40
llm_load_print_meta: n_rot = 128
llm_load_print_meta: n_embd_head_k = 128
llm_load_print_meta: n_embd_head_v = 128
llm_load_print_meta: n_gqa = 1
llm_load_print_meta: n_embd_k_gqa = 5120
llm_load_print_meta: n_embd_v_gqa = 5120
llm_load_print_meta: f_norm_eps = 0.0e+00
llm_load_print_meta: f_norm_rms_eps = 1.0e-05
llm_load_print_meta: f_clamp_kqv = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: n_ff = 13824
llm_load_print_meta: n_expert = 0
llm_load_print_meta: n_expert_used = 0
llm_load_print_meta: rope scaling = linear
llm_load_print_meta: freq_base_train = 10000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_yarn_orig_ctx = 4096
llm_load_print_meta: rope_finetuned = unknown
llm_load_print_meta: model type = 13B
llm_load_print_meta: model ftype = Q4_0
llm_load_print_meta: model params = 13.02 B
llm_load_print_meta: model size = 6.86 GiB (4.53 BPW)
llm_load_print_meta: general.name = LLaMA v2
llm_load_print_meta: BOS token = 1 '<s>'
llm_load_print_meta: EOS token = 2 '</s>'
llm_load_print_meta: UNK token = 0 '<unk>'
llm_load_print_meta: LF token = 13 '<0x0A>'
llm_load_tensors: ggml ctx size = 0.14 MiB
llm_load_tensors: using ROCm for GPU acceleration
llm_load_tensors: system memory used = 88.03 MiB
llm_load_tensors: VRAM used = 6936.01 MiB
llm_load_tensors: offloading 40 repeating layers to GPU
llm_load_tensors: offloading non-repeating layers to GPU
llm_load_tensors: offloaded 41/41 layers to GPU
...................................................................................................
llama_new_context_with_model: n_ctx = 2048
llama_new_context_with_model: freq_base = 10000.0
llama_new_context_with_model: freq_scale = 1
llama_kv_cache_init: VRAM kv self = 1600.00 MB
llama_new_context_with_model: KV self size = 1600.00 MiB, K (f16): 800.00 MiB, V (f16): 800.00 MiB
llama_build_graph: non-view tensors processed: 844/844
llama_new_context_with_model: compute buffer total size = 197.19 MiB
llama_new_context_with_model: VRAM scratch buffer: 194.00 MiB
llama_new_context_with_model: total VRAM used: 8730.01 MiB (model: 6936.01 MiB, context: 1794.00 MiB)
CUDA error: shared object initialization failed
current device: 0, in function ggml_cuda_op_flatten at /go/src/github.com/jmorganca/ollama/llm/llama.cpp/ggml-cuda.cu:8688
hipGetLastError()
GGML_ASSERT: /go/src/github.com/jmorganca/ollama/llm/llama.cpp/ggml-cuda.cu:229: !"CUDA error"
loading library /tmp/ollama2441091586/rocm_v6/libext_server.so
No symbol table is loaded. Use the "file" command.
ptrace: Operation not permitted.
No stack.
The program is not being run.
SIGABRT: abort
PC=0x7fb4b251d387 m=31 sigcode=18446744073709551610
signal arrived during cgo execution
goroutine 66 [syscall]:
runtime.cgocall(0x9b4670, 0xc00055e808)
/usr/local/go/src/runtime/cgocall.go:157 +0x4b fp=0xc00055e7e0 sp=0xc00055e7a8 pc=0x409b0b
github.com/jmorganca/ollama/llm._Cfunc_dyn_llama_server_init({0x7fb410000e00, 0x7fb409a545a0, 0x7fb409a54cf0, 0x7fb409a54d80, 0x7fb409a54f30, 0x7fb409a550a0, 0x7fb409a55560, 0x7fb409a55540, 0x7fb409a555f0, 0x7fb409a55ba0, ...}, ...)
_cgo_gotypes.go:280 +0x45 fp=0xc00055e808 sp=0xc00055e7e0 pc=0x7c2b25
github.com/jmorganca/ollama/llm.newDynExtServer.func7(0xae6f99?, 0x62?)
/go/src/github.com/jmorganca/ollama/llm/dyn_ext_server.go:142 +0xef fp=0xc00055e8f8 sp=0xc00055e808 pc=0x7c3fcf
github.com/jmorganca/ollama/llm.newDynExtServer({0xc00002a840, 0x2e}, {0xc0000302a0, _}, {_, _, _}, {0x0, 0x0, 0x0}, ...)
/go/src/github.com/jmorganca/ollama/llm/dyn_ext_server.go:142 +0xa32 fp=0xc00055eb88 sp=0xc00055e8f8 pc=0x7c3cd2
github.com/jmorganca/ollama/llm.newLlmServer({{_, _, _}, {_, _}, {_, _}}, {_, _}, {0x0, ...}, ...)
/go/src/github.com/jmorganca/ollama/llm/llm.go:147 +0x36a fp=0xc00055ed48 sp=0xc00055eb88 pc=0x7c04ea
github.com/jmorganca/ollama/llm.New({0x0?, 0x1000100000100?}, {0xc0000302a0, _}, {_, _, _}, {0x0, 0x0, 0x0}, ...)
/go/src/github.com/jmorganca/ollama/llm/llm.go:122 +0x6f9 fp=0xc00055efb8 sp=0xc00055ed48 pc=0x7bff19
github.com/jmorganca/ollama/server.load(0xc000002000?, 0xc000002000, {{0x0, 0x800, 0x200, 0x1, 0xffffffffffffffff, 0x0, 0x0, 0x1, ...}, ...}, ...)
/go/src/github.com/jmorganca/ollama/server/routes.go:83 +0x3a5 fp=0xc00055f138 sp=0xc00055efb8 pc=0x9909c5
github.com/jmorganca/ollama/server.ChatHandler(0xc0004a0b00)
/go/src/github.com/jmorganca/ollama/server/routes.go:1071 +0x828 fp=0xc00055f748 sp=0xc00055f138 pc=0x99b308
github.com/gin-gonic/gin.(*Context).Next(...)
/root/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:174
github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func1(0xc0004a0b00)
/go/src/github.com/jmorganca/ollama/server/routes.go:883 +0x68 fp=0xc00055f780 sp=0xc00055f748 pc=0x999e48
github.com/gin-gonic/gin.(*Context).Next(...)
/root/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:174
github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0xc0004a0b00)
/root/go/pkg/mod/github.com/gin-gonic/[email protected]/recovery.go:102 +0x7a fp=0xc00055f7d0 sp=0xc00055f780 pc=0x9756ba
github.com/gin-gonic/gin.(*Context).Next(...)
/root/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:174
github.com/gin-gonic/gin.LoggerWithConfig.func1(0xc0004a0b00)
/root/go/pkg/mod/github.com/gin-gonic/[email protected]/logger.go:240 +0xde fp=0xc00055f980 sp=0xc00055f7d0 pc=0x97485e
github.com/gin-gonic/gin.(*Context).Next(...)
/root/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:174
github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc0005824e0, 0xc0004a0b00)
/root/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:620 +0x65b fp=0xc00055fb08 sp=0xc00055f980 pc=0x97391b
github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc0005824e0, {0x10632140?, 0xc000518540}, 0xc0004a0a00)
/root/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:576 +0x1dd fp=0xc00055fb48 sp=0xc00055fb08 pc=0x9730dd
net/http.serverHandler.ServeHTTP({0x10630460?}, {0x10632140?, 0xc000518540?}, 0x6?)
/usr/local/go/src/net/http/server.go:2938 +0x8e fp=0xc00055fb78 sp=0xc00055fb48 pc=0x6ce60e
net/http.(*conn).serve(0xc0001b4240, {0x106337a8, 0xc0001ec840})
/usr/local/go/src/net/http/server.go:2009 +0x5f4 fp=0xc00055ffb8 sp=0xc00055fb78 pc=0x6ca4f4
net/http.(*Server).Serve.func3()
/usr/local/go/src/net/http/server.go:3086 +0x28 fp=0xc00055ffe0 sp=0xc00055ffb8 pc=0x6cee28
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00055ffe8 sp=0xc00055ffe0 pc=0x46e0a1
created by net/http.(*Server).Serve in goroutine 1
/usr/local/go/src/net/http/server.go:3086 +0x5cb
goroutine 1 [IO wait]:
runtime.gopark(0x4808b0?, 0xc00059d848?, 0x98?, 0xd8?, 0x4f69dd?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00059d828 sp=0xc00059d808 pc=0x43e6ae
runtime.netpollblock(0x46c112?, 0x4092a6?, 0x0?)
/usr/local/go/src/runtime/netpoll.go:564 +0xf7 fp=0xc00059d860 sp=0xc00059d828 pc=0x437137
internal/poll.runtime_pollWait(0x7fb46907be80, 0x72)
/usr/local/go/src/runtime/netpoll.go:343 +0x85 fp=0xc00059d880 sp=0xc00059d860 pc=0x4688c5
internal/poll.(*pollDesc).wait(0xc0004a2000?, 0x4?, 0x0)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc00059d8a8 sp=0xc00059d880 pc=0x4ef627
internal/poll.(*pollDesc).waitRead(...)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc0004a2000)
/usr/local/go/src/internal/poll/fd_unix.go:611 +0x2ac fp=0xc00059d950 sp=0xc00059d8a8 pc=0x4f4b0c
net.(*netFD).accept(0xc0004a2000)
/usr/local/go/src/net/fd_unix.go:172 +0x29 fp=0xc00059da08 sp=0xc00059d950 pc=0x56b609
net.(*TCPListener).accept(0xc0004755a0)
/usr/local/go/src/net/tcpsock_posix.go:152 +0x1e fp=0xc00059da30 sp=0xc00059da08 pc=0x58041e
net.(*TCPListener).Accept(0xc0004755a0)
/usr/local/go/src/net/tcpsock.go:315 +0x30 fp=0xc00059da60 sp=0xc00059da30 pc=0x57f5d0
net/http.(*onceCloseListener).Accept(0xc0001b4240?)
<autogenerated>:1 +0x24 fp=0xc00059da78 sp=0xc00059da60 pc=0x6f13a4
net/http.(*Server).Serve(0xc000122000, {0x10631f30, 0xc0004755a0})
/usr/local/go/src/net/http/server.go:3056 +0x364 fp=0xc00059dba8 sp=0xc00059da78 pc=0x6cea64
github.com/jmorganca/ollama/server.Serve({0x10631f30, 0xc0004755a0})
/go/src/github.com/jmorganca/ollama/server/routes.go:970 +0x488 fp=0xc00059dc98 sp=0xc00059dba8 pc=0x99a328
github.com/jmorganca/ollama/cmd.RunServer(0xc0004a0400?, {0x10a75780?, 0x4?, 0xacee21?})
/go/src/github.com/jmorganca/ollama/cmd/cmd.go:690 +0x199 fp=0xc00059dd30 sp=0xc00059dc98 pc=0x9ac719
github.com/spf13/cobra.(*Command).execute(0xc000453800, {0x10a75780, 0x0, 0x0})
/root/go/pkg/mod/github.com/spf13/[email protected]/command.go:940 +0x87c fp=0xc00059de68 sp=0xc00059dd30 pc=0x7641dc
github.com/spf13/cobra.(*Command).ExecuteC(0xc000452c00)
/root/go/pkg/mod/github.com/spf13/[email protected]/command.go:1068 +0x3a5 fp=0xc00059df20 sp=0xc00059de68 pc=0x764a05
github.com/spf13/cobra.(*Command).Execute(...)
/root/go/pkg/mod/github.com/spf13/[email protected]/command.go:992
github.com/spf13/cobra.(*Command).ExecuteContext(...)
/root/go/pkg/mod/github.com/spf13/[email protected]/command.go:985
main.main()
/go/src/github.com/jmorganca/ollama/main.go:11 +0x4d fp=0xc00059df40 sp=0xc00059df20 pc=0x9b378d
runtime.main()
/usr/local/go/src/runtime/proc.go:267 +0x2bb fp=0xc00059dfe0 sp=0xc00059df40 pc=0x43e25b
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00059dfe8 sp=0xc00059dfe0 pc=0x46e0a1
goroutine 2 [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000090fa8 sp=0xc000090f88 pc=0x43e6ae
runtime.goparkunlock(...)
/usr/local/go/src/runtime/proc.go:404
runtime.forcegchelper()
/usr/local/go/src/runtime/proc.go:322 +0xb3 fp=0xc000090fe0 sp=0xc000090fa8 pc=0x43e533
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000090fe8 sp=0xc000090fe0 pc=0x46e0a1
created by runtime.init.6 in goroutine 1
/usr/local/go/src/runtime/proc.go:310 +0x1a
goroutine 3 [GC sweep wait]:
runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000091778 sp=0xc000091758 pc=0x43e6ae
runtime.goparkunlock(...)
/usr/local/go/src/runtime/proc.go:404
runtime.bgsweep(0x0?)
/usr/local/go/src/runtime/mgcsweep.go:321 +0xdf fp=0xc0000917c8 sp=0xc000091778 pc=0x42a5ff
runtime.gcenable.func1()
/usr/local/go/src/runtime/mgc.go:200 +0x25 fp=0xc0000917e0 sp=0xc0000917c8 pc=0x41f725
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000917e8 sp=0xc0000917e0 pc=0x46e0a1
created by runtime.gcenable in goroutine 1
/usr/local/go/src/runtime/mgc.go:200 +0x66
goroutine 4 [GC scavenge wait]:
runtime.gopark(0x3572e7?, 0x7a2aec?, 0x0?, 0x0?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000091f70 sp=0xc000091f50 pc=0x43e6ae
runtime.goparkunlock(...)
/usr/local/go/src/runtime/proc.go:404
runtime.(*scavengerState).park(0x10a45b00)
/usr/local/go/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc000091fa0 sp=0xc000091f70 pc=0x427e29
runtime.bgscavenge(0x0?)
/usr/local/go/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc000091fc8 sp=0xc000091fa0 pc=0x4283d9
runtime.gcenable.func2()
/usr/local/go/src/runtime/mgc.go:201 +0x25 fp=0xc000091fe0 sp=0xc000091fc8 pc=0x41f6c5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000091fe8 sp=0xc000091fe0 pc=0x46e0a1
created by runtime.gcenable in goroutine 1
/usr/local/go/src/runtime/mgc.go:201 +0xa5
goroutine 5 [finalizer wait]:
runtime.gopark(0xac7de0?, 0x10043f801?, 0x0?, 0x0?, 0x446865?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000090628 sp=0xc000090608 pc=0x43e6ae
runtime.runfinq()
/usr/local/go/src/runtime/mfinal.go:193 +0x107 fp=0xc0000907e0 sp=0xc000090628 pc=0x41e7a7
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000907e8 sp=0xc0000907e0 pc=0x46e0a1
created by runtime.createfing in goroutine 1
/usr/local/go/src/runtime/mfinal.go:163 +0x3d
goroutine 6 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8a03f?, 0x3?, 0xf0?, 0x0?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000092750 sp=0xc000092730 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0000927e0 sp=0xc000092750 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000927e8 sp=0xc0000927e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 18 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8a053?, 0x3?, 0x94?, 0x60?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008c750 sp=0xc00008c730 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008c7e0 sp=0xc00008c750 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008c7e8 sp=0xc00008c7e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 19 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af81473?, 0x1?, 0x89?, 0x78?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008cf50 sp=0xc00008cf30 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008cfe0 sp=0xc00008cf50 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008cfe8 sp=0xc00008cfe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 34 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af89f80?, 0x3?, 0x86?, 0x77?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000508750 sp=0xc000508730 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0005087e0 sp=0xc000508750 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005087e8 sp=0xc0005087e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 20 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8a0fd?, 0x1?, 0x29?, 0x17?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008d750 sp=0xc00008d730 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008d7e0 sp=0xc00008d750 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008d7e8 sp=0xc00008d7e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 35 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8aab2?, 0x3?, 0x9b?, 0xa5?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000508f50 sp=0xc000508f30 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000508fe0 sp=0xc000508f50 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000508fe8 sp=0xc000508fe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 7 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8e277?, 0x3?, 0xc9?, 0x93?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000092f50 sp=0xc000092f30 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000092fe0 sp=0xc000092f50 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000092fe8 sp=0xc000092fe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 36 [GC worker (idle)]:
runtime.gopark(0xc000037228?, 0x1?, 0xb5?, 0xa4?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000509750 sp=0xc000509730 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0005097e0 sp=0xc000509750 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005097e8 sp=0xc0005097e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 8 [GC worker (idle)]:
runtime.gopark(0x10a774a0?, 0x3?, 0x23?, 0xe5?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000093750 sp=0xc000093730 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0000937e0 sp=0xc000093750 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000937e8 sp=0xc0000937e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 9 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af813d3?, 0x3?, 0xfc?, 0x64?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000093f50 sp=0xc000093f30 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000093fe0 sp=0xc000093f50 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000093fe8 sp=0xc000093fe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 21 [GC worker (idle)]:
runtime.gopark(0x10a774a0?, 0x3?, 0xbd?, 0x50?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008df50 sp=0xc00008df30 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008dfe0 sp=0xc00008df50 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008dfe8 sp=0xc00008dfe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 22 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8ae9c?, 0x3?, 0x9c?, 0xad?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008e750 sp=0xc00008e730 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008e7e0 sp=0xc00008e750 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008e7e8 sp=0xc00008e7e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 37 [GC worker (idle)]:
runtime.gopark(0x10a774a0?, 0x1?, 0xee?, 0x2c?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000509f50 sp=0xc000509f30 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000509fe0 sp=0xc000509f50 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000509fe8 sp=0xc000509fe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 23 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8affa?, 0xc00046e4e0?, 0x1a?, 0x14?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008ef50 sp=0xc00008ef30 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008efe0 sp=0xc00008ef50 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008efe8 sp=0xc00008efe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 38 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8c527?, 0x3?, 0x5c?, 0x68?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00050a750 sp=0xc00050a730 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00050a7e0 sp=0xc00050a750 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00050a7e8 sp=0xc00050a7e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 39 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af7e3ba?, 0x3?, 0x53?, 0x3?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00050af50 sp=0xc00050af30 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00050afe0 sp=0xc00050af50 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00050afe8 sp=0xc00050afe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 24 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8ce59?, 0x3?, 0xd0?, 0xa8?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008f750 sp=0xc00008f730 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008f7e0 sp=0xc00008f750 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008f7e8 sp=0xc00008f7e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 10 [GC worker (idle)]:
runtime.gopark(0x10a774a0?, 0x1?, 0x59?, 0x4c?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000504750 sp=0xc000504730 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0005047e0 sp=0xc000504750 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005047e8 sp=0xc0005047e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 25 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8c834?, 0x3?, 0x37?, 0x44?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008ff50 sp=0xc00008ff30 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008ffe0 sp=0xc00008ff50 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008ffe8 sp=0xc00008ffe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 26 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8e186?, 0x1?, 0xa5?, 0x89?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000118750 sp=0xc000118730 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0001187e0 sp=0xc000118750 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0001187e8 sp=0xc0001187e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 40 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8c9cf?, 0x1?, 0x9c?, 0xec?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00050b750 sp=0xc00050b730 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00050b7e0 sp=0xc00050b750 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00050b7e8 sp=0xc00050b7e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 11 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8a175?, 0x3?, 0xa4?, 0x3d?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000504f50 sp=0xc000504f30 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000504fe0 sp=0xc000504f50 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000504fe8 sp=0xc000504fe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 12 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8cb6a?, 0x3?, 0xd1?, 0xff?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000505750 sp=0xc000505730 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0005057e0 sp=0xc000505750 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005057e8 sp=0xc0005057e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 13 [GC worker (idle)]:
runtime.gopark(0x10a774a0?, 0x1?, 0x5d?, 0x34?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000505f50 sp=0xc000505f30 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000505fe0 sp=0xc000505f50 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000505fe8 sp=0xc000505fe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 14 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8cf90?, 0x3?, 0xd7?, 0x7b?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000506750 sp=0xc000506730 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0005067e0 sp=0xc000506750 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005067e8 sp=0xc0005067e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 41 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8921e?, 0x3?, 0x63?, 0xf5?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00050bf50 sp=0xc00050bf30 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00050bfe0 sp=0xc00050bf50 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00050bfe8 sp=0xc00050bfe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 27 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8cb74?, 0x3?, 0xb6?, 0xb1?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000118f50 sp=0xc000118f30 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000118fe0 sp=0xc000118f50 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000118fe8 sp=0xc000118fe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 42 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8cd18?, 0x3?, 0x7a?, 0x70?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000114750 sp=0xc000114730 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0001147e0 sp=0xc000114750 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0001147e8 sp=0xc0001147e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 15 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8750a?, 0x3?, 0x9b?, 0xc3?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000506f50 sp=0xc000506f30 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000506fe0 sp=0xc000506f50 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000506fe8 sp=0xc000506fe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 28 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8cb7e?, 0x3?, 0x67?, 0x79?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000119750 sp=0xc000119730 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0001197e0 sp=0xc000119750 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0001197e8 sp=0xc0001197e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 16 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8cb42?, 0x1?, 0xdc?, 0xa5?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000507750 sp=0xc000507730 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0005077e0 sp=0xc000507750 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005077e8 sp=0xc0005077e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 29 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8bd35?, 0x3?, 0x2d?, 0xb8?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000119f50 sp=0xc000119f30 pc=0x43e6ae
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000119fe0 sp=0xc000119f50 pc=0x4212a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000119fe8 sp=0xc000119fe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/local/go/src/runtime/mgc.go:1217 +0x1c
goroutine 30 [select, locked to thread]:
runtime.gopark(0xc000114fa8?, 0x2?, 0x49?, 0xe9?, 0xc000114fa4?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000114e38 sp=0xc000114e18 pc=0x43e6ae
runtime.selectgo(0xc000114fa8, 0xc000114fa0, 0x0?, 0x0, 0x0?, 0x1)
/usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc000114f58 sp=0xc000114e38 pc=0x44e1e5
runtime.ensureSigM.func1()
/usr/local/go/src/runtime/signal_unix.go:1014 +0x19f fp=0xc000114fe0 sp=0xc000114f58 pc=0x46521f
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000114fe8 sp=0xc000114fe0 pc=0x46e0a1
created by runtime.ensureSigM in goroutine 1
/usr/local/go/src/runtime/signal_unix.go:997 +0xc8
goroutine 50 [syscall]:
runtime.notetsleepg(0x0?, 0x0?)
/usr/local/go/src/runtime/lock_futex.go:236 +0x29 fp=0xc0005947a0 sp=0xc000594768 pc=0x411209
os/signal.signal_recv()
/usr/local/go/src/runtime/sigqueue.go:152 +0x29 fp=0xc0005947c0 sp=0xc0005947a0 pc=0x46aa69
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:23 +0x13 fp=0xc0005947e0 sp=0xc0005947c0 pc=0x6f3dd3
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005947e8 sp=0xc0005947e0 pc=0x46e0a1
created by os/signal.Notify.func1.1 in goroutine 1
/usr/local/go/src/os/signal/signal.go:151 +0x1f
goroutine 51 [chan receive]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000594f18 sp=0xc000594ef8 pc=0x43e6ae
runtime.chanrecv(0xc00068e840, 0x0, 0x1)
/usr/local/go/src/runtime/chan.go:583 +0x3cd fp=0xc000594f90 sp=0xc000594f18 pc=0x40beed
runtime.chanrecv1(0x0?, 0x0?)
/usr/local/go/src/runtime/chan.go:442 +0x12 fp=0xc000594fb8 sp=0xc000594f90 pc=0x40baf2
github.com/jmorganca/ollama/server.Serve.func1()
/go/src/github.com/jmorganca/ollama/server/routes.go:952 +0x25 fp=0xc000594fe0 sp=0xc000594fb8 pc=0x99a3c5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000594fe8 sp=0xc000594fe0 pc=0x46e0a1
created by github.com/jmorganca/ollama/server.Serve in goroutine 1
/go/src/github.com/jmorganca/ollama/server/routes.go:951 +0x3f6
goroutine 67 [IO wait]:
runtime.gopark(0x0?, 0xb?, 0x0?, 0x0?, 0x11?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000115da0 sp=0xc000115d80 pc=0x43e6ae
runtime.netpollblock(0x47ea18?, 0x4092a6?, 0x0?)
/usr/local/go/src/runtime/netpoll.go:564 +0xf7 fp=0xc000115dd8 sp=0xc000115da0 pc=0x437137
internal/poll.runtime_pollWait(0x7fb46907bc90, 0x72)
/usr/local/go/src/runtime/netpoll.go:343 +0x85 fp=0xc000115df8 sp=0xc000115dd8 pc=0x4688c5
internal/poll.(*pollDesc).wait(0xc0001c0600?, 0xc0001eca01?, 0x0)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc000115e20 sp=0xc000115df8 pc=0x4ef627
internal/poll.(*pollDesc).waitRead(...)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0001c0600, {0xc0001eca01, 0x1, 0x1})
/usr/local/go/src/internal/poll/fd_unix.go:164 +0x27a fp=0xc000115eb8 sp=0xc000115e20 pc=0x4f091a
net.(*netFD).Read(0xc0001c0600, {0xc0001eca01?, 0x0?, 0x0?})
/usr/local/go/src/net/fd_posix.go:55 +0x25 fp=0xc000115f00 sp=0xc000115eb8 pc=0x5695e5
net.(*conn).Read(0xc000690060, {0xc0001eca01?, 0x0?, 0x0?})
/usr/local/go/src/net/net.go:179 +0x45 fp=0xc000115f48 sp=0xc000115f00 pc=0x577885
net.(*TCPConn).Read(0x0?, {0xc0001eca01?, 0x0?, 0x0?})
<autogenerated>:1 +0x25 fp=0xc000115f78 sp=0xc000115f48 pc=0x589785
net/http.(*connReader).backgroundRead(0xc0001ec9f0)
/usr/local/go/src/net/http/server.go:683 +0x37 fp=0xc000115fc8 sp=0xc000115f78 pc=0x6c4377
net/http.(*connReader).startBackgroundRead.func2()
/usr/local/go/src/net/http/server.go:679 +0x25 fp=0xc000115fe0 sp=0xc000115fc8 pc=0x6c42a5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000115fe8 sp=0xc000115fe0 pc=0x46e0a1
created by net/http.(*connReader).startBackgroundRead in goroutine 66
/usr/local/go/src/net/http/server.go:679 +0xba
rax 0x0
rbx 0x7fb409c0950e
rcx 0x7fb4b251d387
rdx 0x6
rdi 0x1
rsi 0x24
rbp 0x21f0
rsp 0x7fb41effc368
r8 0x0
r9 0x1
r10 0x8
r11 0x202
r12 0x7fb4b28af868
r13 0x7fb0f380a1b0
r14 0x7fb409c08c1c
r15 0x7fb409c094b3
rip 0x7fb4b251d387
rflags 0x202
cs 0x33
fs 0x0
gs 0x0
GPU: RX 7900 XTX RAM: 64GB Model: llama2:13b
I'm seeing the exact same error stack. I built it with
go generate ./... && go build -ldflags '-linkmode external -extldflags "-static"' -o .
GO 1.21.6 GPU NVIDIA A2 15GB Model llama2
@xlmnxp you seem to have hit #2054 which is fixed in 0.1.22. We've split out ROCm support into a separate image due to the size which is tagged ollama/ollama:0.1.22-rocm
@ThatOneCalculator from the log excerpt, I can't quite tell if you're hitting the same problem of iGPUs causing problems. We just merged the fix for that a few hours ago, so it might be worth rebasing and see if that fixes your problem. If not, can you run with OLLAMA_DEBUG=1 set and share the early log lines so we can see a bit more?
@mrisher23 given you're on an NVIDIA card and not Radeon I would expect a different scenario - can you share the log, or open a new issue?
I'll try again right now. I doubt it since I don't even have an iGPU...
Nope, still crashed. Here's the output of env HOME=/var/lib/ollama HCC_AMDGPU_TARGET=gfx1030 OLLAMA_ORIGINS="*" HSA_OVERRIDE_GFX_VERSION=10.3.0 ROCM_PATH=/opt/rocm OLLAMA_DEBUG=1 ./ollama serve and attempting to load tinyllama
time=2024-01-26T17:50:25.794-08:00 level=DEBUG source=/home/kainoa/.local/share/ollama-build/server/routes.go:939 msg="Debug logging enabled"
time=2024-01-26T17:50:25.794-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/server/images.go:857 msg="total blobs: 37"
time=2024-01-26T17:50:25.794-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/server/images.go:864 msg="total unused blobs removed: 0"
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.
[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
- using env: export GIN_MODE=release
- using code: gin.SetMode(gin.ReleaseMode)
[GIN-debug] POST /api/pull --> github.com/jmorganca/ollama/server.PullModelHandler (5 handlers)
[GIN-debug] POST /api/generate --> github.com/jmorganca/ollama/server.GenerateHandler (5 handlers)
[GIN-debug] POST /api/chat --> github.com/jmorganca/ollama/server.ChatHandler (5 handlers)
[GIN-debug] POST /api/embeddings --> github.com/jmorganca/ollama/server.EmbeddingHandler (5 handlers)
[GIN-debug] POST /api/create --> github.com/jmorganca/ollama/server.CreateModelHandler (5 handlers)
[GIN-debug] POST /api/push --> github.com/jmorganca/ollama/server.PushModelHandler (5 handlers)
[GIN-debug] POST /api/copy --> github.com/jmorganca/ollama/server.CopyModelHandler (5 handlers)
[GIN-debug] DELETE /api/delete --> github.com/jmorganca/ollama/server.DeleteModelHandler (5 handlers)
[GIN-debug] POST /api/show --> github.com/jmorganca/ollama/server.ShowModelHandler (5 handlers)
[GIN-debug] POST /api/blobs/:digest --> github.com/jmorganca/ollama/server.CreateBlobHandler (5 handlers)
[GIN-debug] HEAD /api/blobs/:digest --> github.com/jmorganca/ollama/server.HeadBlobHandler (5 handlers)
[GIN-debug] GET / --> github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
[GIN-debug] GET /api/tags --> github.com/jmorganca/ollama/server.ListModelsHandler (5 handlers)
[GIN-debug] GET /api/version --> github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func3 (5 handlers)
[GIN-debug] HEAD / --> github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
[GIN-debug] HEAD /api/tags --> github.com/jmorganca/ollama/server.ListModelsHandler (5 handlers)
[GIN-debug] HEAD /api/version --> github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func3 (5 handlers)
time=2024-01-26T17:50:25.795-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/server/routes.go:963 msg="Listening on 127.0.0.1:11434 (version 0.0.0)"
time=2024-01-26T17:50:25.795-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/llm/payload_common.go:106 msg="Extracting dynamic libraries..."
time=2024-01-26T17:50:25.845-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/llm/payload_common.go:145 msg="Dynamic LLM libraries [rocm_v5 cpu cpu_avx cpu_avx2]"
time=2024-01-26T17:50:25.845-08:00 level=DEBUG source=/home/kainoa/.local/share/ollama-build/llm/payload_common.go:146 msg="Override detection logic by setting OLLAMA_LLM_LIBRARY"
time=2024-01-26T17:50:25.845-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/gpu/gpu.go:94 msg="Detecting GPU type"
time=2024-01-26T17:50:25.845-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/gpu/gpu.go:242 msg="Searching for GPU management library libnvidia-ml.so"
time=2024-01-26T17:50:25.845-08:00 level=DEBUG source=/home/kainoa/.local/share/ollama-build/gpu/gpu.go:260 msg="gpu management search paths: [/usr/local/cuda/lib64/libnvidia-ml.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libnvidia-ml.so* /usr/lib/x86_64-linux-gnu/libnvidia-ml.so* /usr/lib/wsl/lib/libnvidia-ml.so* /usr/lib/wsl/drivers/*/libnvidia-ml.so* /opt/cuda/lib64/libnvidia-ml.so* /usr/lib*/libnvidia-ml.so* /usr/local/lib*/libnvidia-ml.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libnvidia-ml.so* /usr/lib/aarch64-linux-gnu/libnvidia-ml.so* /opt/cuda/targets/x86_64-linux/lib/stubs/libnvidia-ml.so* /home/kainoa/.local/share/ollama-build/libnvidia-ml.so* /home/kainoa/.local/lib/mojo/libnvidia-ml.so*]"
time=2024-01-26T17:50:25.852-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/gpu/gpu.go:288 msg="Discovered GPU libraries: []"
time=2024-01-26T17:50:25.852-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/gpu/gpu.go:242 msg="Searching for GPU management library librocm_smi64.so"
time=2024-01-26T17:50:25.852-08:00 level=DEBUG source=/home/kainoa/.local/share/ollama-build/gpu/gpu.go:260 msg="gpu management search paths: [/opt/rocm*/lib*/librocm_smi64.so* /home/kainoa/.local/share/ollama-build/librocm_smi64.so* /home/kainoa/.local/lib/mojo/librocm_smi64.so*]"
time=2024-01-26T17:50:25.852-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/gpu/gpu.go:288 msg="Discovered GPU libraries: [/opt/rocm/lib/librocm_smi64.so.5.0 /opt/rocm-bak/lib/librocm_smi64.so.5.0]"
wiring rocm management library functions in /opt/rocm/lib/librocm_smi64.so.5.0
dlsym: rsmi_init
dlsym: rsmi_shut_down
dlsym: rsmi_dev_memory_total_get
dlsym: rsmi_dev_memory_usage_get
dlsym: rsmi_version_get
dlsym: rsmi_num_monitor_devices
dlsym: rsmi_dev_id_get
dlsym: rsmi_dev_name_get
dlsym: rsmi_dev_brand_get
dlsym: rsmi_dev_vendor_name_get
dlsym: rsmi_dev_vram_vendor_get
dlsym: rsmi_dev_serial_number_get
dlsym: rsmi_dev_subsystem_name_get
dlsym: rsmi_dev_vbios_version_get
time=2024-01-26T17:50:25.855-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/gpu/gpu.go:109 msg="Radeon GPU detected"
time=2024-01-26T17:50:25.855-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/gpu/cpu_common.go:11 msg="CPU has AVX2"
discovered 1 ROCm GPU Devices
[0] ROCm device name: Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]
[0] ROCm brand: Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]
[0] ROCm vendor: Advanced Micro Devices, Inc. [AMD/ATI]
[0] ROCm VRAM vendor: samsung
rsmi_dev_serial_number_get failed: 2
[0] ROCm subsystem name: 0x2331
[0] ROCm vbios version: 113-D51221-R67XTE
[0] ROCm totalMem 12868124672
[0] ROCm usedMem 700297216
time=2024-01-26T17:50:25.857-08:00 level=DEBUG source=/home/kainoa/.local/share/ollama-build/gpu/gpu.go:231 msg="rocm detected 1 devices with 10443M available memory"
[GIN] 2024/01/26 - 17:50:29 | 200 | 42.3µs | 127.0.0.1 | HEAD "/"
[GIN] 2024/01/26 - 17:50:29 | 200 | 429.656µs | 127.0.0.1 | POST "/api/show"
[GIN] 2024/01/26 - 17:50:29 | 200 | 149.602µs | 127.0.0.1 | POST "/api/show"
time=2024-01-26T17:50:29.247-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/gpu/cpu_common.go:11 msg="CPU has AVX2"
discovered 1 ROCm GPU Devices
[0] ROCm device name: Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]
[0] ROCm brand: Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]
[0] ROCm vendor: Advanced Micro Devices, Inc. [AMD/ATI]
[0] ROCm VRAM vendor: samsung
rsmi_dev_serial_number_get failed: 2
[0] ROCm subsystem name: 0x2331
[0] ROCm vbios version: 113-D51221-R67XTE
[0] ROCm totalMem 12868124672
[0] ROCm usedMem 691302400
time=2024-01-26T17:50:29.250-08:00 level=DEBUG source=/home/kainoa/.local/share/ollama-build/gpu/gpu.go:231 msg="rocm detected 1 devices with 10451M available memory"
time=2024-01-26T17:50:29.250-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/gpu/cpu_common.go:11 msg="CPU has AVX2"
discovered 1 ROCm GPU Devices
[0] ROCm device name: Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]
[0] ROCm brand: Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]
[0] ROCm vendor: Advanced Micro Devices, Inc. [AMD/ATI]
[0] ROCm VRAM vendor: samsung
rsmi_dev_serial_number_get failed: 2
[0] ROCm subsystem name: 0x2331
[0] ROCm vbios version: 113-D51221-R67XTE
[0] ROCm totalMem 12868124672
[0] ROCm usedMem 691302400
time=2024-01-26T17:50:29.252-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/gpu/cpu_common.go:11 msg="CPU has AVX2"
loading library /tmp/ollama2063163931/rocm_v5/libext_server.so
time=2024-01-26T17:50:29.289-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/llm/dyn_ext_server.go:90 msg="Loading Dynamic llm server: /tmp/ollama2063163931/rocm_v5/libext_server.so"
time=2024-01-26T17:50:29.289-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/llm/dyn_ext_server.go:145 msg="Initializing llama server"
[1706320229] system info: AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 |
[1706320229] Performing pre-initialization of GPU
ggml_init_cublas: GGML_CUDA_FORCE_MMQ: no
ggml_init_cublas: CUDA_USE_TENSOR_CORES: yes
ggml_init_cublas: found 1 ROCm devices:
Device 0: AMD Radeon RX 6700 XT, compute capability 10.3, VMM: no
llama_model_loader: loaded meta data with 23 key-value pairs and 201 tensors from /var/lib/ollama/.ollama/models/blobs/sha256:2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816 (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = llama
llama_model_loader: - kv 1: general.name str = TinyLlama
llama_model_loader: - kv 2: llama.context_length u32 = 2048
llama_model_loader: - kv 3: llama.embedding_length u32 = 2048
llama_model_loader: - kv 4: llama.block_count u32 = 22
llama_model_loader: - kv 5: llama.feed_forward_length u32 = 5632
llama_model_loader: - kv 6: llama.rope.dimension_count u32 = 64
llama_model_loader: - kv 7: llama.attention.head_count u32 = 32
llama_model_loader: - kv 8: llama.attention.head_count_kv u32 = 4
llama_model_loader: - kv 9: llama.attention.layer_norm_rms_epsilon f32 = 0.000010
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
llama_model_loader: - kv 11: general.file_type u32 = 2
llama_model_loader: - kv 12: tokenizer.ggml.model str = llama
llama_model_loader: - kv 13: tokenizer.ggml.tokens arr[str,32000] = ["<unk>", "<s>", "</s>", "<0x00>", "<...
llama_model_loader: - kv 14: tokenizer.ggml.scores arr[f32,32000] = [0.000000, 0.000000, 0.000000, 0.0000...
llama_model_loader: - kv 15: tokenizer.ggml.token_type arr[i32,32000] = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...
llama_model_loader: - kv 16: tokenizer.ggml.merges arr[str,61249] = ["▁ t", "e r", "i n", "▁ a", "e n...
llama_model_loader: - kv 17: tokenizer.ggml.bos_token_id u32 = 1
llama_model_loader: - kv 18: tokenizer.ggml.eos_token_id u32 = 2
llama_model_loader: - kv 19: tokenizer.ggml.unknown_token_id u32 = 0
llama_model_loader: - kv 20: tokenizer.ggml.padding_token_id u32 = 2
llama_model_loader: - kv 21: tokenizer.chat_template str = {% for message in messages %}\n{% if m...
llama_model_loader: - kv 22: general.quantization_version u32 = 2
llama_model_loader: - type f32: 45 tensors
llama_model_loader: - type q4_0: 155 tensors
llama_model_loader: - type q6_K: 1 tensors
llm_load_vocab: special tokens definition check successful ( 259/32000 ).
llm_load_print_meta: format = GGUF V3 (latest)
llm_load_print_meta: arch = llama
llm_load_print_meta: vocab type = SPM
llm_load_print_meta: n_vocab = 32000
llm_load_print_meta: n_merges = 0
llm_load_print_meta: n_ctx_train = 2048
llm_load_print_meta: n_embd = 2048
llm_load_print_meta: n_head = 32
llm_load_print_meta: n_head_kv = 4
llm_load_print_meta: n_layer = 22
llm_load_print_meta: n_rot = 64
llm_load_print_meta: n_embd_head_k = 64
llm_load_print_meta: n_embd_head_v = 64
llm_load_print_meta: n_gqa = 8
llm_load_print_meta: n_embd_k_gqa = 256
llm_load_print_meta: n_embd_v_gqa = 256
llm_load_print_meta: f_norm_eps = 0.0e+00
llm_load_print_meta: f_norm_rms_eps = 1.0e-05
llm_load_print_meta: f_clamp_kqv = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: n_ff = 5632
llm_load_print_meta: n_expert = 0
llm_load_print_meta: n_expert_used = 0
llm_load_print_meta: rope scaling = linear
llm_load_print_meta: freq_base_train = 10000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_yarn_orig_ctx = 2048
llm_load_print_meta: rope_finetuned = unknown
llm_load_print_meta: model type = 1B
llm_load_print_meta: model ftype = Q4_0
llm_load_print_meta: model params = 1.10 B
llm_load_print_meta: model size = 606.53 MiB (4.63 BPW)
llm_load_print_meta: general.name = TinyLlama
llm_load_print_meta: BOS token = 1 '<s>'
llm_load_print_meta: EOS token = 2 '</s>'
llm_load_print_meta: UNK token = 0 '<unk>'
llm_load_print_meta: PAD token = 2 '</s>'
llm_load_print_meta: LF token = 13 '<0x0A>'
llm_load_tensors: ggml ctx size = 0.15 MiB
llm_load_tensors: offloading 22 repeating layers to GPU
llm_load_tensors: offloading non-repeating layers to GPU
llm_load_tensors: offloaded 23/23 layers to GPU
llm_load_tensors: ROCm0 buffer size = 571.37 MiB
llm_load_tensors: CPU buffer size = 35.16 MiB
.......................................................................................
llama_new_context_with_model: n_ctx = 2048
llama_new_context_with_model: freq_base = 10000.0
llama_new_context_with_model: freq_scale = 1
llama_kv_cache_init: ROCm0 KV buffer size = 44.00 MiB
llama_new_context_with_model: KV self size = 44.00 MiB, K (f16): 22.00 MiB, V (f16): 22.00 MiB
llama_new_context_with_model: ROCm_Host input buffer size = 8.01 MiB
llama_new_context_with_model: ROCm0 compute buffer size = 144.00 MiB
llama_new_context_with_model: ROCm_Host compute buffer size = 4.00 MiB
llama_new_context_with_model: graph splits (measure): 3
[1706320230] warming up the model with an empty run
SIGSEGV: segmentation violation
PC=0x70af3512b380 m=9 sigcode=128 addr=0x0
signal arrived during cgo execution
goroutine 67 gp=0xc0005028c0 m=9 mp=0xc000580008 [syscall]:
runtime.cgocall(0x9d2c10, 0xc0003ae838)
/usr/lib/go/src/runtime/cgocall.go:157 +0x4b fp=0xc0003ae810 sp=0xc0003ae7d8 pc=0x40a72b
github.com/jmorganca/ollama/llm._Cfunc_dyn_llama_server_init({0x70af18016510, 0x70af366d3310, 0x70af366d3b50, 0x70af366d3be0, 0x70af366d3d90, 0x70af366d3f10, 0x70af366d4440, 0x70af366d4420, 0x70af366d44d0, 0x70af366d49b0, ...}, ...)
_cgo_gotypes.go:290 +0x45 fp=0xc0003ae838 sp=0xc0003ae810 pc=0x7e0585
github.com/jmorganca/ollama/llm.newDynExtServer.func7(0xc0000ac4b0, 0xc000013530)
/home/kainoa/.local/share/ollama-build/llm/dyn_ext_server.go:148 +0x112 fp=0xc0003ae978 sp=0xc0003ae838 pc=0x7e1bb2
github.com/jmorganca/ollama/llm.newDynExtServer({0xc00048a2a0, 0x2e}, {0xc0005b6a10, _}, {_, _, _}, {0x0, 0x0, 0x0}, ...)
/home/kainoa/.local/share/ollama-build/llm/dyn_ext_server.go:148 +0xac5 fp=0xc0003aebc0 sp=0xc0003ae978 pc=0x7e17e5
github.com/jmorganca/ollama/llm.newLlmServer({{_, _, _}, {_, _}, {_, _}}, {_, _}, {0x0, ...}, ...)
/home/kainoa/.local/share/ollama-build/llm/llm.go:148 +0x405 fp=0xc0003aed80 sp=0xc0003aebc0 pc=0x7dddc5
github.com/jmorganca/ollama/llm.New({0x0?, 0x0?}, {0xc0005b6a10, _}, {_, _, _}, {0x0, 0x0, 0x0}, ...)
/home/kainoa/.local/share/ollama-build/llm/llm.go:123 +0x755 fp=0xc0003aeff0 sp=0xc0003aed80 pc=0x7dd775
github.com/jmorganca/ollama/server.load(0xc00057e000, 0xc000002f00, {{0x0, 0x800, 0x200, 0x1, 0xffffffffffffffff, 0x0, 0x0, 0x1, ...}, ...}, ...)
/home/kainoa/.local/share/ollama-build/server/routes.go:83 +0x3a9 fp=0xc0003af160 sp=0xc0003aeff0 pc=0x9ade09
github.com/jmorganca/ollama/server.ChatHandler(0xc00057e000)
/home/kainoa/.local/share/ollama-build/server/routes.go:1098 +0x637 fp=0xc0003af770 sp=0xc0003af160 pc=0x9b8857
github.com/gin-gonic/gin.(*Context).Next(...)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:174
github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func1(0xc00057e000)
/home/kainoa/.local/share/ollama-build/server/routes.go:903 +0x68 fp=0xc0003af7a8 sp=0xc0003af770 pc=0x9b74c8
github.com/gin-gonic/gin.(*Context).Next(...)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:174
github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0xc00057e000)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/recovery.go:102 +0x7a fp=0xc0003af7f8 sp=0xc0003af7a8 pc=0x991bfa
github.com/gin-gonic/gin.(*Context).Next(...)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:174
github.com/gin-gonic/gin.LoggerWithConfig.func1(0xc00057e000)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/logger.go:240 +0xdd fp=0xc0003af9a8 sp=0xc0003af7f8 pc=0x990d3d
github.com/gin-gonic/gin.(*Context).Next(...)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:174
github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc0000efd40, 0xc00057e000)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:620 +0x66e fp=0xc0003afb28 sp=0xc0003af9a8 pc=0x99022e
github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc0000efd40, {0x12d8aa0, 0xc00019e2a0}, 0xc000465320)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:576 +0x1b2 fp=0xc0003afb60 sp=0xc0003afb28 pc=0x98f9f2
net/http.serverHandler.ServeHTTP({0x12d6dc0?}, {0x12d8aa0?, 0xc00019e2a0?}, 0x6?)
/usr/lib/go/src/net/http/server.go:3137 +0x8e fp=0xc0003afb90 sp=0xc0003afb60 pc=0x6e89ce
net/http.(*conn).serve(0xc000478090, {0x12da0e8, 0xc0001cd0e0})
/usr/lib/go/src/net/http/server.go:2039 +0x5e8 fp=0xc0003affb8 sp=0xc0003afb90 pc=0x6e3d88
net/http.(*Server).Serve.gowrap3()
/usr/lib/go/src/net/http/server.go:3285 +0x28 fp=0xc0003affe0 sp=0xc0003affb8 pc=0x6e91e8
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0003affe8 sp=0xc0003affe0 pc=0x473ca1
created by net/http.(*Server).Serve in goroutine 1
/usr/lib/go/src/net/http/server.go:3285 +0x4b4
goroutine 1 gp=0xc0000061c0 m=nil [IO wait]:
runtime.gopark(0xc000050f08?, 0xc0000438b0?, 0x71?, 0xd5?, 0x2000?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc0004dd830 sp=0xc0004dd810 pc=0x4411ce
runtime.netpollblock(0xc0000438c8?, 0x409ec6?, 0x0?)
/usr/lib/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc0004dd868 sp=0xc0004dd830 pc=0x439fd7
internal/poll.runtime_pollWait(0x70af9d6866d0, 0x72)
/usr/lib/go/src/runtime/netpoll.go:345 +0x85 fp=0xc0004dd888 sp=0xc0004dd868 pc=0x46e3a5
internal/poll.(*pollDesc).wait(0x4?, 0xe0?, 0x0)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0004dd8b0 sp=0xc0004dd888 pc=0x4f7767
internal/poll.(*pollDesc).waitRead(...)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc000482300)
/usr/lib/go/src/internal/poll/fd_unix.go:611 +0x2ac fp=0xc0004dd958 sp=0xc0004dd8b0 pc=0x4fcb0c
net.(*netFD).accept(0xc000482300)
/usr/lib/go/src/net/fd_unix.go:172 +0x29 fp=0xc0004dda10 sp=0xc0004dd958 pc=0x576b89
net.(*TCPListener).accept(0xc0004577c0)
/usr/lib/go/src/net/tcpsock_posix.go:159 +0x1e fp=0xc0004dda38 sp=0xc0004dda10 pc=0x58be5e
net.(*TCPListener).Accept(0xc0004577c0)
/usr/lib/go/src/net/tcpsock.go:327 +0x30 fp=0xc0004dda68 sp=0xc0004dda38 pc=0x58b050
net/http.(*onceCloseListener).Accept(0xc000478090?)
<autogenerated>:1 +0x24 fp=0xc0004dda80 sp=0xc0004dda68 pc=0x70b3a4
net/http.(*Server).Serve(0xc000390ff0, {0x12d8830, 0xc0004577c0})
/usr/lib/go/src/net/http/server.go:3255 +0x33e fp=0xc0004ddbb0 sp=0xc0004dda80 pc=0x6e8dfe
github.com/jmorganca/ollama/server.Serve({0x12d8830, 0xc0004577c0})
/home/kainoa/.local/share/ollama-build/server/routes.go:990 +0x517 fp=0xc0004ddcc0 sp=0xc0004ddbb0 pc=0x9b7a37
github.com/jmorganca/ollama/cmd.RunServer(0xc000486400?, {0x176b740?, 0x4?, 0xaf0ddb?})
/home/kainoa/.local/share/ollama-build/cmd/cmd.go:692 +0x199 fp=0xc0004ddd58 sp=0xc0004ddcc0 pc=0x9c9e39
github.com/spf13/cobra.(*Command).execute(0xc000480f08, {0x176b740, 0x0, 0x0})
/home/kainoa/go/pkg/mod/github.com/spf13/[email protected]/command.go:940 +0x882 fp=0xc0004dde78 sp=0xc0004ddd58 pc=0x77dea2
github.com/spf13/cobra.(*Command).ExecuteC(0xc000480308)
/home/kainoa/go/pkg/mod/github.com/spf13/[email protected]/command.go:1068 +0x3a5 fp=0xc0004ddf30 sp=0xc0004dde78 pc=0x77e6e5
github.com/spf13/cobra.(*Command).Execute(...)
/home/kainoa/go/pkg/mod/github.com/spf13/[email protected]/command.go:992
github.com/spf13/cobra.(*Command).ExecuteContext(...)
/home/kainoa/go/pkg/mod/github.com/spf13/[email protected]/command.go:985
main.main()
/home/kainoa/.local/share/ollama-build/main.go:11 +0x4d fp=0xc0004ddf50 sp=0xc0004ddf30 pc=0x9d1d2d
runtime.main()
/usr/lib/go/src/runtime/proc.go:271 +0x29d fp=0xc0004ddfe0 sp=0xc0004ddf50 pc=0x440d9d
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0004ddfe8 sp=0xc0004ddfe0 pc=0x473ca1
goroutine 2 gp=0xc000006c40 m=nil [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000078fa8 sp=0xc000078f88 pc=0x4411ce
runtime.goparkunlock(...)
/usr/lib/go/src/runtime/proc.go:408
runtime.forcegchelper()
/usr/lib/go/src/runtime/proc.go:326 +0xb3 fp=0xc000078fe0 sp=0xc000078fa8 pc=0x441053
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000078fe8 sp=0xc000078fe0 pc=0x473ca1
created by runtime.init.6 in goroutine 1
/usr/lib/go/src/runtime/proc.go:314 +0x1a
goroutine 3 gp=0xc000007180 m=nil [GC sweep wait]:
runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000079780 sp=0xc000079760 pc=0x4411ce
runtime.goparkunlock(...)
/usr/lib/go/src/runtime/proc.go:408
runtime.bgsweep(0xc0000380e0)
/usr/lib/go/src/runtime/mgcsweep.go:318 +0xdf fp=0xc0000797c8 sp=0xc000079780 pc=0x42c81f
runtime.gcenable.gowrap1()
/usr/lib/go/src/runtime/mgc.go:203 +0x25 fp=0xc0000797e0 sp=0xc0000797c8 pc=0x421105
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000797e8 sp=0xc0000797e0 pc=0x473ca1
created by runtime.gcenable in goroutine 1
/usr/lib/go/src/runtime/mgc.go:203 +0x66
goroutine 4 gp=0xc000007340 m=nil [GC scavenge wait]:
runtime.gopark(0x14b098?, 0x3b9aca00?, 0x0?, 0x0?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000079f78 sp=0xc000079f58 pc=0x4411ce
runtime.goparkunlock(...)
/usr/lib/go/src/runtime/proc.go:408
runtime.(*scavengerState).park(0x1709c60)
/usr/lib/go/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc000079fa8 sp=0xc000079f78 pc=0x42a1a9
runtime.bgscavenge(0xc0000380e0)
/usr/lib/go/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc000079fc8 sp=0xc000079fa8 pc=0x42a759
runtime.gcenable.gowrap2()
/usr/lib/go/src/runtime/mgc.go:204 +0x25 fp=0xc000079fe0 sp=0xc000079fc8 pc=0x4210a5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000079fe8 sp=0xc000079fe0 pc=0x473ca1
created by runtime.gcenable in goroutine 1
/usr/lib/go/src/runtime/mgc.go:204 +0xa5
goroutine 5 gp=0xc000007c00 m=nil [finalizer wait]:
runtime.gopark(0xc000078648?, 0x4144c5?, 0xa8?, 0x1?, 0xaea740?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000078620 sp=0xc000078600 pc=0x4411ce
runtime.runfinq()
/usr/lib/go/src/runtime/mfinal.go:194 +0x107 fp=0xc0000787e0 sp=0xc000078620 pc=0x420147
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000787e8 sp=0xc0000787e0 pc=0x473ca1
created by runtime.createfing in goroutine 1
/usr/lib/go/src/runtime/mfinal.go:164 +0x3d
goroutine 6 gp=0xc000398e00 m=nil [select, locked to thread]:
runtime.gopark(0xc00007a7a8?, 0x2?, 0x69?, 0x14?, 0xc00007a794?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc00007a638 sp=0xc00007a618 pc=0x4411ce
runtime.selectgo(0xc00007a7a8, 0xc00007a790, 0x0?, 0x0, 0x0?, 0x1)
/usr/lib/go/src/runtime/select.go:327 +0x725 fp=0xc00007a758 sp=0xc00007a638 pc=0x4524e5
runtime.ensureSigM.func1()
/usr/lib/go/src/runtime/signal_unix.go:1034 +0x19f fp=0xc00007a7e0 sp=0xc00007a758 pc=0x46b0ff
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00007a7e8 sp=0xc00007a7e0 pc=0x473ca1
created by runtime.ensureSigM in goroutine 1
/usr/lib/go/src/runtime/signal_unix.go:1017 +0xc8
goroutine 18 gp=0xc000102380 m=3 mp=0xc00007f008 [syscall]:
runtime.notetsleepg(0x176c300, 0xffffffffffffffff)
/usr/lib/go/src/runtime/lock_futex.go:246 +0x29 fp=0xc0000747a0 sp=0xc000074778 pc=0x412ae9
os/signal.signal_recv()
/usr/lib/go/src/runtime/sigqueue.go:152 +0x29 fp=0xc0000747c0 sp=0xc0000747a0 pc=0x470709
os/signal.loop()
/usr/lib/go/src/os/signal/signal_unix.go:23 +0x13 fp=0xc0000747e0 sp=0xc0000747c0 pc=0x70d753
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000747e8 sp=0xc0000747e0 pc=0x473ca1
created by os/signal.Notify.func1.1 in goroutine 1
/usr/lib/go/src/os/signal/signal.go:151 +0x1f
goroutine 34 gp=0xc000502380 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000514750 sp=0xc000514730 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0005147e0 sp=0xc000514750 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0005147e8 sp=0xc0005147e0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 7 gp=0xc000398fc0 m=nil [GC worker (idle)]:
runtime.gopark(0xa6974ed7e6?, 0x3?, 0x51?, 0x37?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc00007af50 sp=0xc00007af30 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00007afe0 sp=0xc00007af50 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00007afe8 sp=0xc00007afe0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 8 gp=0xc000399180 m=nil [GC worker (idle)]:
runtime.gopark(0xa6974ecb16?, 0x1?, 0x82?, 0x46?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc00007b750 sp=0xc00007b730 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00007b7e0 sp=0xc00007b750 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00007b7e8 sp=0xc00007b7e0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 9 gp=0xc000399340 m=nil [GC worker (idle)]:
runtime.gopark(0x176d5a0?, 0x3?, 0x9e?, 0x11?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc00007bf50 sp=0xc00007bf30 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00007bfe0 sp=0xc00007bf50 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00007bfe8 sp=0xc00007bfe0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 10 gp=0xc000399500 m=nil [GC worker (idle)]:
runtime.gopark(0xa69750b8cd?, 0x1?, 0x5e?, 0x1a?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000510750 sp=0xc000510730 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0005107e0 sp=0xc000510750 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0005107e8 sp=0xc0005107e0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 11 gp=0xc0003996c0 m=nil [GC worker (idle)]:
runtime.gopark(0xa6974ecb52?, 0x3?, 0x2?, 0x17?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000510f50 sp=0xc000510f30 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000510fe0 sp=0xc000510f50 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000510fe8 sp=0xc000510fe0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 12 gp=0xc000399880 m=nil [GC worker (idle)]:
runtime.gopark(0xa6974eca80?, 0x1?, 0xca?, 0xc5?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000511750 sp=0xc000511730 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0005117e0 sp=0xc000511750 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0005117e8 sp=0xc0005117e0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 13 gp=0xc000399a40 m=nil [GC worker (idle)]:
runtime.gopark(0xa6974ed73c?, 0x3?, 0x65?, 0x37?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000511f50 sp=0xc000511f30 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000511fe0 sp=0xc000511f50 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000511fe8 sp=0xc000511fe0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 14 gp=0xc000399c00 m=nil [GC worker (idle)]:
runtime.gopark(0xa69750c755?, 0x3?, 0x26?, 0x1b?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000512750 sp=0xc000512730 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0005127e0 sp=0xc000512750 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0005127e8 sp=0xc0005127e0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 15 gp=0xc000399dc0 m=nil [GC worker (idle)]:
runtime.gopark(0xa6974ed0b6?, 0x1?, 0x91?, 0x1d?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000512f50 sp=0xc000512f30 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000512fe0 sp=0xc000512f50 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000512fe8 sp=0xc000512fe0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 16 gp=0xc0004b4000 m=nil [GC worker (idle)]:
runtime.gopark(0x176d5a0?, 0x1?, 0x82?, 0x3c?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000513750 sp=0xc000513730 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0005137e0 sp=0xc000513750 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0005137e8 sp=0xc0005137e0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 50 gp=0xc0004b41c0 m=nil [GC worker (idle)]:
runtime.gopark(0xa6974ed7f0?, 0x3?, 0xbb?, 0x3e?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000513f50 sp=0xc000513f30 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000513fe0 sp=0xc000513f50 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000513fe8 sp=0xc000513fe0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 51 gp=0xc0004b4380 m=nil [GC worker (idle)]:
runtime.gopark(0xa6974ed91c?, 0x3?, 0xec?, 0x4?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc0004ba750 sp=0xc0004ba730 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0004ba7e0 sp=0xc0004ba750 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0004ba7e8 sp=0xc0004ba7e0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 52 gp=0xc0004b4540 m=nil [GC worker (idle)]:
runtime.gopark(0xa6974eba36?, 0x3?, 0x62?, 0x43?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc0004baf50 sp=0xc0004baf30 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0004bafe0 sp=0xc0004baf50 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0004bafe8 sp=0xc0004bafe0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 19 gp=0xc0001028c0 m=nil [GC worker (idle)]:
runtime.gopark(0xa6974ecb66?, 0x3?, 0xd4?, 0x35?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000074f50 sp=0xc000074f30 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000074fe0 sp=0xc000074f50 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000074fe8 sp=0xc000074fe0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 20 gp=0xc000102a80 m=nil [GC worker (idle)]:
runtime.gopark(0xa6974edc32?, 0x1?, 0x83?, 0x69?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000075750 sp=0xc000075730 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000757e0 sp=0xc000075750 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000757e8 sp=0xc0000757e0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 21 gp=0xc000102c40 m=nil [chan receive]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000075f18 sp=0xc000075ef8 pc=0x4411ce
runtime.chanrecv(0xc0004ae660, 0x0, 0x1)
/usr/lib/go/src/runtime/chan.go:583 +0x3bf fp=0xc000075f90 sp=0xc000075f18 pc=0x40cd3f
runtime.chanrecv1(0x0?, 0x0?)
/usr/lib/go/src/runtime/chan.go:442 +0x12 fp=0xc000075fb8 sp=0xc000075f90 pc=0x40c952
github.com/jmorganca/ollama/server.Serve.func1()
/home/kainoa/.local/share/ollama-build/server/routes.go:972 +0x25 fp=0xc000075fe0 sp=0xc000075fb8 pc=0x9b7ac5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000075fe8 sp=0xc000075fe0 pc=0x473ca1
created by github.com/jmorganca/ollama/server.Serve in goroutine 1
/home/kainoa/.local/share/ollama-build/server/routes.go:971 +0x458
goroutine 57 gp=0xc000502a80 m=nil [IO wait]:
runtime.gopark(0x75?, 0xc0004df958?, 0x40?, 0xf9?, 0xb?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc0004df910 sp=0xc0004df8f0 pc=0x4411ce
runtime.netpollblock(0x4851d8?, 0x409ec6?, 0x0?)
/usr/lib/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc0004df948 sp=0xc0004df910 pc=0x439fd7
internal/poll.runtime_pollWait(0x70af9d6865d8, 0x72)
/usr/lib/go/src/runtime/netpoll.go:345 +0x85 fp=0xc0004df968 sp=0xc0004df948 pc=0x46e3a5
internal/poll.(*pollDesc).wait(0xc000482900?, 0xc0001b6000?, 0x0)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0004df990 sp=0xc0004df968 pc=0x4f7767
internal/poll.(*pollDesc).waitRead(...)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000482900, {0xc0001b6000, 0x1000, 0x1000})
/usr/lib/go/src/internal/poll/fd_unix.go:164 +0x27a fp=0xc0004dfa28 sp=0xc0004df990 pc=0x4f8a5a
net.(*netFD).Read(0xc000482900, {0xc0001b6000?, 0xc0004dfa98?, 0x4f7c25?})
/usr/lib/go/src/net/fd_posix.go:55 +0x25 fp=0xc0004dfa70 sp=0xc0004dfa28 pc=0x574ba5
net.(*conn).Read(0xc0001180c0, {0xc0001b6000?, 0x0?, 0xc0001cd3b8?})
/usr/lib/go/src/net/net.go:179 +0x45 fp=0xc0004dfab8 sp=0xc0004dfa70 pc=0x582da5
net.(*TCPConn).Read(0xc0001cd3b0?, {0xc0001b6000?, 0xc000482900?, 0xc0004dfaf0?})
<autogenerated>:1 +0x25 fp=0xc0004dfae8 sp=0xc0004dfab8 pc=0x594425
net/http.(*connReader).Read(0xc0001cd3b0, {0xc0001b6000, 0x1000, 0x1000})
/usr/lib/go/src/net/http/server.go:789 +0x14b fp=0xc0004dfb38 sp=0xc0004dfae8 pc=0x6de18b
bufio.(*Reader).fill(0xc0004ae720)
/usr/lib/go/src/bufio/bufio.go:110 +0x103 fp=0xc0004dfb70 sp=0xc0004dfb38 pc=0x665243
bufio.(*Reader).Peek(0xc0004ae720, 0x4)
/usr/lib/go/src/bufio/bufio.go:148 +0x53 fp=0xc0004dfb90 sp=0xc0004dfb70 pc=0x665373
net/http.(*conn).serve(0xc00019c2d0, {0x12da0e8, 0xc0001cd0e0})
/usr/lib/go/src/net/http/server.go:2074 +0x749 fp=0xc0004dffb8 sp=0xc0004dfb90 pc=0x6e3ee9
net/http.(*Server).Serve.gowrap3()
/usr/lib/go/src/net/http/server.go:3285 +0x28 fp=0xc0004dffe0 sp=0xc0004dffb8 pc=0x6e91e8
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0004dffe8 sp=0xc0004dffe0 pc=0x473ca1
created by net/http.(*Server).Serve in goroutine 1
/usr/lib/go/src/net/http/server.go:3285 +0x4b4
goroutine 22 gp=0xc000502c40 m=nil [IO wait]:
runtime.gopark(0x51e?, 0xc0003b3958?, 0x40?, 0x39?, 0xb?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000593910 sp=0xc0005938f0 pc=0x4411ce
runtime.netpollblock(0x4851d8?, 0x409ec6?, 0x0?)
/usr/lib/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc000593948 sp=0xc000593910 pc=0x439fd7
internal/poll.runtime_pollWait(0x70af9d6864e0, 0x72)
/usr/lib/go/src/runtime/netpoll.go:345 +0x85 fp=0xc000593968 sp=0xc000593948 pc=0x46e3a5
internal/poll.(*pollDesc).wait(0xc000434000?, 0xc000496000?, 0x0)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc000593990 sp=0xc000593968 pc=0x4f7767
internal/poll.(*pollDesc).waitRead(...)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000434000, {0xc000496000, 0x1000, 0x1000})
/usr/lib/go/src/internal/poll/fd_unix.go:164 +0x27a fp=0xc000593a28 sp=0xc000593990 pc=0x4f8a5a
net.(*netFD).Read(0xc000434000, {0xc000496000?, 0xc0003b3a98?, 0x4f7c25?})
/usr/lib/go/src/net/fd_posix.go:55 +0x25 fp=0xc000593a70 sp=0xc000593a28 pc=0x574ba5
net.(*conn).Read(0xc00007c000, {0xc000496000?, 0x0?, 0xc0003ea188?})
/usr/lib/go/src/net/net.go:179 +0x45 fp=0xc000593ab8 sp=0xc000593a70 pc=0x582da5
net.(*TCPConn).Read(0xc0003ea180?, {0xc000496000?, 0xc000434000?, 0xc0003b3af0?})
<autogenerated>:1 +0x25 fp=0xc000593ae8 sp=0xc000593ab8 pc=0x594425
net/http.(*connReader).Read(0xc0003ea180, {0xc000496000, 0x1000, 0x1000})
/usr/lib/go/src/net/http/server.go:789 +0x14b fp=0xc000593b38 sp=0xc000593ae8 pc=0x6de18b
bufio.(*Reader).fill(0xc0001ac8a0)
/usr/lib/go/src/bufio/bufio.go:110 +0x103 fp=0xc000593b70 sp=0xc000593b38 pc=0x665243
bufio.(*Reader).Peek(0xc0001ac8a0, 0x4)
/usr/lib/go/src/bufio/bufio.go:148 +0x53 fp=0xc000593b90 sp=0xc000593b70 pc=0x665373
net/http.(*conn).serve(0xc000478000, {0x12da0e8, 0xc0001cd0e0})
/usr/lib/go/src/net/http/server.go:2074 +0x749 fp=0xc000593fb8 sp=0xc000593b90 pc=0x6e3ee9
net/http.(*Server).Serve.gowrap3()
/usr/lib/go/src/net/http/server.go:3285 +0x28 fp=0xc000593fe0 sp=0xc000593fb8 pc=0x6e91e8
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000593fe8 sp=0xc000593fe0 pc=0x473ca1
created by net/http.(*Server).Serve in goroutine 1
/usr/lib/go/src/net/http/server.go:3285 +0x4b4
goroutine 59 gp=0xc0004b4a80 m=nil [IO wait]:
runtime.gopark(0x10?, 0x10?, 0xf0?, 0xcd?, 0xb?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc00059cda8 sp=0xc00059cd88 pc=0x4411ce
runtime.netpollblock(0x4851d8?, 0x409ec6?, 0x0?)
/usr/lib/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc00059cde0 sp=0xc00059cda8 pc=0x439fd7
internal/poll.runtime_pollWait(0x70af9d6863e8, 0x72)
/usr/lib/go/src/runtime/netpoll.go:345 +0x85 fp=0xc00059ce00 sp=0xc00059cde0 pc=0x46e3a5
internal/poll.(*pollDesc).wait(0xc000434080?, 0xc0004a3061?, 0x0)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc00059ce28 sp=0xc00059ce00 pc=0x4f7767
internal/poll.(*pollDesc).waitRead(...)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000434080, {0xc0004a3061, 0x1, 0x1})
/usr/lib/go/src/internal/poll/fd_unix.go:164 +0x27a fp=0xc00059cec0 sp=0xc00059ce28 pc=0x4f8a5a
net.(*netFD).Read(0xc000434080, {0xc0004a3061?, 0xc00059cf48?, 0x470410?})
/usr/lib/go/src/net/fd_posix.go:55 +0x25 fp=0xc00059cf08 sp=0xc00059cec0 pc=0x574ba5
net.(*conn).Read(0xc000118000, {0xc0004a3061?, 0x0?, 0x176b740?})
/usr/lib/go/src/net/net.go:179 +0x45 fp=0xc00059cf50 sp=0xc00059cf08 pc=0x582da5
net.(*TCPConn).Read(0x16a21d0?, {0xc0004a3061?, 0x0?, 0x0?})
<autogenerated>:1 +0x25 fp=0xc00059cf80 sp=0xc00059cf50 pc=0x594425
net/http.(*connReader).backgroundRead(0xc0004a3050)
/usr/lib/go/src/net/http/server.go:681 +0x37 fp=0xc00059cfc8 sp=0xc00059cf80 pc=0x6ddcf7
net/http.(*connReader).startBackgroundRead.gowrap2()
/usr/lib/go/src/net/http/server.go:677 +0x25 fp=0xc00059cfe0 sp=0xc00059cfc8 pc=0x6ddc25
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00059cfe8 sp=0xc00059cfe0 pc=0x473ca1
created by net/http.(*connReader).startBackgroundRead in goroutine 67
/usr/lib/go/src/net/http/server.go:677 +0xba
rax 0x0
rbx 0x70ac6adef0a0
rcx 0x70af0c300480
rdx 0x4ac
rdi 0x70af0c300480
rsi 0x70ac6accd590
rbp 0x70af4d5fddf0
rsp 0x70af4d5fdbc0
r8 0x2c
r9 0x1
r10 0x3
r11 0x70af1892b390
r12 0x8
r13 0x70af1892b390
r14 0x70ac6acca070
r15 0x70ac6acca228
rip 0x70af3512b380
rflags 0x10246
cs 0x33
fs 0x0
gs 0x0
Ignore deleted comment about AVX2. Still get a crash, built with CLBlast_DIR=/usr/lib/cmake/CLBlast AMDGPU_TARGETS="gfx1030" ROCM_PATH=/opt/rocm OLLAMA_CUSTOM_CPU_DEFS="-DLLAMA_AVX=on -DLLAMA_AVX2=off" go generate ./... && go build .
time=2024-01-26T18:12:39.403-08:00 level=DEBUG source=/home/kainoa/.local/share/ollama-build/server/routes.go:939 msg="Debug logging enabled"
time=2024-01-26T18:12:39.403-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/server/images.go:857 msg="total blobs: 37"
time=2024-01-26T18:12:39.403-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/server/images.go:864 msg="total unused blobs removed: 0"
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.
[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
- using env: export GIN_MODE=release
- using code: gin.SetMode(gin.ReleaseMode)
[GIN-debug] POST /api/pull --> github.com/jmorganca/ollama/server.PullModelHandler (5 handlers)
[GIN-debug] POST /api/generate --> github.com/jmorganca/ollama/server.GenerateHandler (5 handlers)
[GIN-debug] POST /api/chat --> github.com/jmorganca/ollama/server.ChatHandler (5 handlers)
[GIN-debug] POST /api/embeddings --> github.com/jmorganca/ollama/server.EmbeddingHandler (5 handlers)
[GIN-debug] POST /api/create --> github.com/jmorganca/ollama/server.CreateModelHandler (5 handlers)
[GIN-debug] POST /api/push --> github.com/jmorganca/ollama/server.PushModelHandler (5 handlers)
[GIN-debug] POST /api/copy --> github.com/jmorganca/ollama/server.CopyModelHandler (5 handlers)
[GIN-debug] DELETE /api/delete --> github.com/jmorganca/ollama/server.DeleteModelHandler (5 handlers)
[GIN-debug] POST /api/show --> github.com/jmorganca/ollama/server.ShowModelHandler (5 handlers)
[GIN-debug] POST /api/blobs/:digest --> github.com/jmorganca/ollama/server.CreateBlobHandler (5 handlers)
[GIN-debug] HEAD /api/blobs/:digest --> github.com/jmorganca/ollama/server.HeadBlobHandler (5 handlers)
[GIN-debug] GET / --> github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
[GIN-debug] GET /api/tags --> github.com/jmorganca/ollama/server.ListModelsHandler (5 handlers)
[GIN-debug] GET /api/version --> github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func3 (5 handlers)
[GIN-debug] HEAD / --> github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
[GIN-debug] HEAD /api/tags --> github.com/jmorganca/ollama/server.ListModelsHandler (5 handlers)
[GIN-debug] HEAD /api/version --> github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func3 (5 handlers)
time=2024-01-26T18:12:39.403-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/server/routes.go:963 msg="Listening on 127.0.0.1:11434 (version 0.0.0)"
time=2024-01-26T18:12:39.403-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/llm/payload_common.go:106 msg="Extracting dynamic libraries..."
time=2024-01-26T18:12:39.422-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/llm/payload_common.go:145 msg="Dynamic LLM libraries [cpu_avx cpu rocm_v5 cpu_avx2]"
time=2024-01-26T18:12:39.422-08:00 level=DEBUG source=/home/kainoa/.local/share/ollama-build/llm/payload_common.go:146 msg="Override detection logic by setting OLLAMA_LLM_LIBRARY"
time=2024-01-26T18:12:39.422-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/gpu/gpu.go:94 msg="Detecting GPU type"
time=2024-01-26T18:12:39.422-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/gpu/gpu.go:242 msg="Searching for GPU management library libnvidia-ml.so"
time=2024-01-26T18:12:39.422-08:00 level=DEBUG source=/home/kainoa/.local/share/ollama-build/gpu/gpu.go:260 msg="gpu management search paths: [/usr/local/cuda/lib64/libnvidia-ml.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libnvidia-ml.so* /usr/lib/x86_64-linux-gnu/libnvidia-ml.so* /usr/lib/wsl/lib/libnvidia-ml.so* /usr/lib/wsl/drivers/*/libnvidia-ml.so* /opt/cuda/lib64/libnvidia-ml.so* /usr/lib*/libnvidia-ml.so* /usr/local/lib*/libnvidia-ml.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libnvidia-ml.so* /usr/lib/aarch64-linux-gnu/libnvidia-ml.so* /opt/cuda/targets/x86_64-linux/lib/stubs/libnvidia-ml.so* /home/kainoa/.local/share/ollama-build/libnvidia-ml.so* /home/kainoa/.local/lib/mojo/libnvidia-ml.so*]"
time=2024-01-26T18:12:39.429-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/gpu/gpu.go:288 msg="Discovered GPU libraries: []"
time=2024-01-26T18:12:39.429-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/gpu/gpu.go:242 msg="Searching for GPU management library librocm_smi64.so"
time=2024-01-26T18:12:39.429-08:00 level=DEBUG source=/home/kainoa/.local/share/ollama-build/gpu/gpu.go:260 msg="gpu management search paths: [/opt/rocm*/lib*/librocm_smi64.so* /home/kainoa/.local/share/ollama-build/librocm_smi64.so* /home/kainoa/.local/lib/mojo/librocm_smi64.so*]"
time=2024-01-26T18:12:39.429-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/gpu/gpu.go:288 msg="Discovered GPU libraries: [/opt/rocm/lib/librocm_smi64.so.5.0 /opt/rocm-bak/lib/librocm_smi64.so.5.0]"
wiring rocm management library functions in /opt/rocm/lib/librocm_smi64.so.5.0
dlsym: rsmi_init
dlsym: rsmi_shut_down
dlsym: rsmi_dev_memory_total_get
dlsym: rsmi_dev_memory_usage_get
dlsym: rsmi_version_get
dlsym: rsmi_num_monitor_devices
dlsym: rsmi_dev_id_get
dlsym: rsmi_dev_name_get
dlsym: rsmi_dev_brand_get
dlsym: rsmi_dev_vendor_name_get
dlsym: rsmi_dev_vram_vendor_get
dlsym: rsmi_dev_serial_number_get
dlsym: rsmi_dev_subsystem_name_get
dlsym: rsmi_dev_vbios_version_get
time=2024-01-26T18:12:39.432-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/gpu/gpu.go:109 msg="Radeon GPU detected"
time=2024-01-26T18:12:39.432-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/gpu/cpu_common.go:11 msg="CPU has AVX2"
discovered 1 ROCm GPU Devices
[0] ROCm device name: Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]
[0] ROCm brand: Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]
[0] ROCm vendor: Advanced Micro Devices, Inc. [AMD/ATI]
[0] ROCm VRAM vendor: samsung
rsmi_dev_serial_number_get failed: 2
[0] ROCm subsystem name: 0x2331
[0] ROCm vbios version: 113-D51221-R67XTE
[0] ROCm totalMem 12868124672
[0] ROCm usedMem 758726656
time=2024-01-26T18:12:39.434-08:00 level=DEBUG source=/home/kainoa/.local/share/ollama-build/gpu/gpu.go:231 msg="rocm detected 1 devices with 10393M available memory"
[GIN] 2024/01/26 - 18:12:42 | 200 | 31.28µs | 127.0.0.1 | HEAD "/"
[GIN] 2024/01/26 - 18:12:42 | 200 | 308.15µs | 127.0.0.1 | POST "/api/show"
[GIN] 2024/01/26 - 18:12:42 | 200 | 145.38µs | 127.0.0.1 | POST "/api/show"
time=2024-01-26T18:12:42.592-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/gpu/cpu_common.go:11 msg="CPU has AVX2"
discovered 1 ROCm GPU Devices
[0] ROCm device name: Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]
[0] ROCm brand: Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]
[0] ROCm vendor: Advanced Micro Devices, Inc. [AMD/ATI]
[0] ROCm VRAM vendor: samsung
rsmi_dev_serial_number_get failed: 2
[0] ROCm subsystem name: 0x2331
[0] ROCm vbios version: 113-D51221-R67XTE
[0] ROCm totalMem 12868124672
[0] ROCm usedMem 732635136
time=2024-01-26T18:12:42.594-08:00 level=DEBUG source=/home/kainoa/.local/share/ollama-build/gpu/gpu.go:231 msg="rocm detected 1 devices with 10415M available memory"
time=2024-01-26T18:12:42.594-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/gpu/cpu_common.go:11 msg="CPU has AVX2"
discovered 1 ROCm GPU Devices
[0] ROCm device name: Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]
[0] ROCm brand: Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]
[0] ROCm vendor: Advanced Micro Devices, Inc. [AMD/ATI]
[0] ROCm VRAM vendor: samsung
rsmi_dev_serial_number_get failed: 2
[0] ROCm subsystem name: 0x2331
[0] ROCm vbios version: 113-D51221-R67XTE
[0] ROCm totalMem 12868124672
[0] ROCm usedMem 732635136
time=2024-01-26T18:12:42.597-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/gpu/cpu_common.go:11 msg="CPU has AVX2"
loading library /tmp/ollama4049440412/rocm_v5/libext_server.so
time=2024-01-26T18:12:42.627-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/llm/dyn_ext_server.go:90 msg="Loading Dynamic llm server: /tmp/ollama4049440412/rocm_v5/libext_server.so"
time=2024-01-26T18:12:42.627-08:00 level=INFO source=/home/kainoa/.local/share/ollama-build/llm/dyn_ext_server.go:145 msg="Initializing llama server"
[1706321562] system info: AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 |
[1706321562] Performing pre-initialization of GPU
ggml_init_cublas: GGML_CUDA_FORCE_MMQ: no
ggml_init_cublas: CUDA_USE_TENSOR_CORES: yes
ggml_init_cublas: found 1 ROCm devices:
Device 0: AMD Radeon RX 6700 XT, compute capability 10.3, VMM: no
llama_model_loader: loaded meta data with 22 key-value pairs and 363 tensors from /var/lib/ollama/.ollama/models/blobs/sha256:444d96c83284ff9812e5935799d00e8116e7884a902afaa25e1c3b6fcddb8111 (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = llama
llama_model_loader: - kv 1: general.name str = LLaMA v2
llama_model_loader: - kv 2: llama.context_length u32 = 4096
llama_model_loader: - kv 3: llama.embedding_length u32 = 4096
llama_model_loader: - kv 4: llama.block_count u32 = 40
llama_model_loader: - kv 5: llama.feed_forward_length u32 = 11008
llama_model_loader: - kv 6: llama.rope.dimension_count u32 = 128
llama_model_loader: - kv 7: llama.attention.head_count u32 = 32
llama_model_loader: - kv 8: llama.attention.head_count_kv u32 = 32
llama_model_loader: - kv 9: llama.attention.layer_norm_rms_epsilon f32 = 0.000010
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
llama_model_loader: - kv 11: general.file_type u32 = 2
llama_model_loader: - kv 12: tokenizer.ggml.model str = llama
llama_model_loader: - kv 13: tokenizer.ggml.tokens arr[str,32000] = ["<unk>", "<s>", "</s>", "<0x00>", "<...
llama_model_loader: - kv 14: tokenizer.ggml.scores arr[f32,32000] = [0.000000, 0.000000, 0.000000, 0.0000...
llama_model_loader: - kv 15: tokenizer.ggml.token_type arr[i32,32000] = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...
llama_model_loader: - kv 16: tokenizer.ggml.bos_token_id u32 = 1
llama_model_loader: - kv 17: tokenizer.ggml.eos_token_id u32 = 2
llama_model_loader: - kv 18: tokenizer.ggml.add_bos_token bool = true
llama_model_loader: - kv 19: tokenizer.ggml.add_eos_token bool = false
llama_model_loader: - kv 20: tokenizer.chat_template str = {% for message in messages %}\n{% if m...
llama_model_loader: - kv 21: general.quantization_version u32 = 2
llama_model_loader: - type f32: 81 tensors
llama_model_loader: - type q4_0: 281 tensors
llama_model_loader: - type q6_K: 1 tensors
llm_load_vocab: special tokens definition check successful ( 259/32000 ).
llm_load_print_meta: format = GGUF V3 (latest)
llm_load_print_meta: arch = llama
llm_load_print_meta: vocab type = SPM
llm_load_print_meta: n_vocab = 32000
llm_load_print_meta: n_merges = 0
llm_load_print_meta: n_ctx_train = 4096
llm_load_print_meta: n_embd = 4096
llm_load_print_meta: n_head = 32
llm_load_print_meta: n_head_kv = 32
llm_load_print_meta: n_layer = 40
llm_load_print_meta: n_rot = 128
llm_load_print_meta: n_embd_head_k = 128
llm_load_print_meta: n_embd_head_v = 128
llm_load_print_meta: n_gqa = 1
llm_load_print_meta: n_embd_k_gqa = 4096
llm_load_print_meta: n_embd_v_gqa = 4096
llm_load_print_meta: f_norm_eps = 0.0e+00
llm_load_print_meta: f_norm_rms_eps = 1.0e-05
llm_load_print_meta: f_clamp_kqv = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: n_ff = 11008
llm_load_print_meta: n_expert = 0
llm_load_print_meta: n_expert_used = 0
llm_load_print_meta: rope scaling = linear
llm_load_print_meta: freq_base_train = 10000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_yarn_orig_ctx = 4096
llm_load_print_meta: rope_finetuned = unknown
llm_load_print_meta: model type = 13B
llm_load_print_meta: model ftype = Q4_0
llm_load_print_meta: model params = 8.36 B
llm_load_print_meta: model size = 4.41 GiB (4.53 BPW)
llm_load_print_meta: general.name = LLaMA v2
llm_load_print_meta: BOS token = 1 '<s>'
llm_load_print_meta: EOS token = 2 '</s>'
llm_load_print_meta: UNK token = 0 '<unk>'
llm_load_print_meta: LF token = 13 '<0x0A>'
llm_load_tensors: ggml ctx size = 0.28 MiB
SIGSEGV: segmentation violation
PC=0x75cf8ab2b380 m=14 sigcode=128 addr=0x0
signal arrived during cgo execution
goroutine 32 gp=0xc00012c540 m=14 mp=0xc00028d808 [syscall]:
runtime.cgocall(0x9d2c10, 0xc000042838)
/usr/lib/go/src/runtime/cgocall.go:157 +0x4b fp=0xc000042810 sp=0xc0000427d8 pc=0x40a72b
github.com/jmorganca/ollama/llm._Cfunc_dyn_llama_server_init({0x75cf44001f10, 0x75cf91410310, 0x75cf91410b50, 0x75cf91410be0, 0x75cf91410d90, 0x75cf91410f10, 0x75cf91411440, 0x75cf91411420, 0x75cf914114d0, 0x75cf914119b0, ...}, ...)
_cgo_gotypes.go:290 +0x45 fp=0xc000042838 sp=0xc000042810 pc=0x7e0585
github.com/jmorganca/ollama/llm.newDynExtServer.func7(0xc0000ac5f0, 0xc0006ca438)
/home/kainoa/.local/share/ollama-build/llm/dyn_ext_server.go:148 +0x112 fp=0xc000042978 sp=0xc000042838 pc=0x7e1bb2
github.com/jmorganca/ollama/llm.newDynExtServer({0xc0005ca000, 0x2e}, {0xc0000385b0, _}, {_, _, _}, {0x0, 0x0, 0x0}, ...)
/home/kainoa/.local/share/ollama-build/llm/dyn_ext_server.go:148 +0xac5 fp=0xc000042bc0 sp=0xc000042978 pc=0x7e17e5
github.com/jmorganca/ollama/llm.newLlmServer({{_, _, _}, {_, _}, {_, _}}, {_, _}, {0x0, ...}, ...)
/home/kainoa/.local/share/ollama-build/llm/llm.go:148 +0x405 fp=0xc000042d80 sp=0xc000042bc0 pc=0x7dddc5
github.com/jmorganca/ollama/llm.New({0x0?, 0x0?}, {0xc0000385b0, _}, {_, _, _}, {0x0, 0x0, 0x0}, ...)
/home/kainoa/.local/share/ollama-build/llm/llm.go:123 +0x755 fp=0xc000042ff0 sp=0xc000042d80 pc=0x7dd775
github.com/jmorganca/ollama/server.load(0xc0001fe000, 0xc000002a80, {{0x0, 0x800, 0x200, 0x1, 0xffffffffffffffff, 0x0, 0x0, 0x1, ...}, ...}, ...)
/home/kainoa/.local/share/ollama-build/server/routes.go:83 +0x3a9 fp=0xc000043160 sp=0xc000042ff0 pc=0x9ade09
github.com/jmorganca/ollama/server.ChatHandler(0xc0001fe000)
/home/kainoa/.local/share/ollama-build/server/routes.go:1098 +0x637 fp=0xc000043770 sp=0xc000043160 pc=0x9b8857
github.com/gin-gonic/gin.(*Context).Next(...)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:174
github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func1(0xc0001fe000)
/home/kainoa/.local/share/ollama-build/server/routes.go:903 +0x68 fp=0xc0000437a8 sp=0xc000043770 pc=0x9b74c8
github.com/gin-gonic/gin.(*Context).Next(...)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:174
github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0xc0001fe000)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/recovery.go:102 +0x7a fp=0xc0000437f8 sp=0xc0000437a8 pc=0x991bfa
github.com/gin-gonic/gin.(*Context).Next(...)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:174
github.com/gin-gonic/gin.LoggerWithConfig.func1(0xc0001fe000)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/logger.go:240 +0xdd fp=0xc0000439a8 sp=0xc0000437f8 pc=0x990d3d
github.com/gin-gonic/gin.(*Context).Next(...)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:174
github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc0004b2000, 0xc0001fe000)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:620 +0x66e fp=0xc000043b28 sp=0xc0000439a8 pc=0x99022e
github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc0004b2000, {0x108bbc0, 0xc00019e1c0}, 0xc000198b40)
/home/kainoa/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:576 +0x1b2 fp=0xc000043b60 sp=0xc000043b28 pc=0x98f9f2
net/http.serverHandler.ServeHTTP({0x1089ee0?}, {0x108bbc0?, 0xc00019e1c0?}, 0x6?)
/usr/lib/go/src/net/http/server.go:3137 +0x8e fp=0xc000043b90 sp=0xc000043b60 pc=0x6e89ce
net/http.(*conn).serve(0xc00019c2d0, {0x108d208, 0xc0005ac690})
/usr/lib/go/src/net/http/server.go:2039 +0x5e8 fp=0xc000043fb8 sp=0xc000043b90 pc=0x6e3d88
net/http.(*Server).Serve.gowrap3()
/usr/lib/go/src/net/http/server.go:3285 +0x28 fp=0xc000043fe0 sp=0xc000043fb8 pc=0x6e91e8
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000043fe8 sp=0xc000043fe0 pc=0x473ca1
created by net/http.(*Server).Serve in goroutine 1
/usr/lib/go/src/net/http/server.go:3285 +0x4b4
goroutine 1 gp=0xc0000061c0 m=nil [IO wait]:
runtime.gopark(0xc00004c508?, 0xc0006f78b0?, 0x71?, 0xd5?, 0x2000?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000353830 sp=0xc000353810 pc=0x4411ce
runtime.netpollblock(0xc0006f78c8?, 0x409ec6?, 0x0?)
/usr/lib/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc000353868 sp=0xc000353830 pc=0x439fd7
internal/poll.runtime_pollWait(0x75cff2c166d0, 0x72)
/usr/lib/go/src/runtime/netpoll.go:345 +0x85 fp=0xc000353888 sp=0xc000353868 pc=0x46e3a5
internal/poll.(*pollDesc).wait(0x4?, 0x27?, 0x0)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0003538b0 sp=0xc000353888 pc=0x4f7767
internal/poll.(*pollDesc).waitRead(...)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc000482300)
/usr/lib/go/src/internal/poll/fd_unix.go:611 +0x2ac fp=0xc000353958 sp=0xc0003538b0 pc=0x4fcb0c
net.(*netFD).accept(0xc000482300)
/usr/lib/go/src/net/fd_unix.go:172 +0x29 fp=0xc000353a10 sp=0xc000353958 pc=0x576b89
net.(*TCPListener).accept(0xc0004557e0)
/usr/lib/go/src/net/tcpsock_posix.go:159 +0x1e fp=0xc000353a38 sp=0xc000353a10 pc=0x58be5e
net.(*TCPListener).Accept(0xc0004557e0)
/usr/lib/go/src/net/tcpsock.go:327 +0x30 fp=0xc000353a68 sp=0xc000353a38 pc=0x58b050
net/http.(*onceCloseListener).Accept(0xc00019c2d0?)
<autogenerated>:1 +0x24 fp=0xc000353a80 sp=0xc000353a68 pc=0x70b3a4
net/http.(*Server).Serve(0xc000390ff0, {0x108b950, 0xc0004557e0})
/usr/lib/go/src/net/http/server.go:3255 +0x33e fp=0xc000353bb0 sp=0xc000353a80 pc=0x6e8dfe
github.com/jmorganca/ollama/server.Serve({0x108b950, 0xc0004557e0})
/home/kainoa/.local/share/ollama-build/server/routes.go:990 +0x517 fp=0xc000353cc0 sp=0xc000353bb0 pc=0x9b7a37
github.com/jmorganca/ollama/cmd.RunServer(0xc000486400?, {0x151e740?, 0x4?, 0xaf0ddb?})
/home/kainoa/.local/share/ollama-build/cmd/cmd.go:692 +0x199 fp=0xc000353d58 sp=0xc000353cc0 pc=0x9c9e39
github.com/spf13/cobra.(*Command).execute(0xc000480f08, {0x151e740, 0x0, 0x0})
/home/kainoa/go/pkg/mod/github.com/spf13/[email protected]/command.go:940 +0x882 fp=0xc000353e78 sp=0xc000353d58 pc=0x77dea2
github.com/spf13/cobra.(*Command).ExecuteC(0xc000480308)
/home/kainoa/go/pkg/mod/github.com/spf13/[email protected]/command.go:1068 +0x3a5 fp=0xc000353f30 sp=0xc000353e78 pc=0x77e6e5
github.com/spf13/cobra.(*Command).Execute(...)
/home/kainoa/go/pkg/mod/github.com/spf13/[email protected]/command.go:992
github.com/spf13/cobra.(*Command).ExecuteContext(...)
/home/kainoa/go/pkg/mod/github.com/spf13/[email protected]/command.go:985
main.main()
/home/kainoa/.local/share/ollama-build/main.go:11 +0x4d fp=0xc000353f50 sp=0xc000353f30 pc=0x9d1d2d
runtime.main()
/usr/lib/go/src/runtime/proc.go:271 +0x29d fp=0xc000353fe0 sp=0xc000353f50 pc=0x440d9d
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000353fe8 sp=0xc000353fe0 pc=0x473ca1
goroutine 2 gp=0xc000006c40 m=nil [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000078fa8 sp=0xc000078f88 pc=0x4411ce
runtime.goparkunlock(...)
/usr/lib/go/src/runtime/proc.go:408
runtime.forcegchelper()
/usr/lib/go/src/runtime/proc.go:326 +0xb3 fp=0xc000078fe0 sp=0xc000078fa8 pc=0x441053
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000078fe8 sp=0xc000078fe0 pc=0x473ca1
created by runtime.init.6 in goroutine 1
/usr/lib/go/src/runtime/proc.go:314 +0x1a
goroutine 3 gp=0xc000007180 m=nil [GC sweep wait]:
runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000079780 sp=0xc000079760 pc=0x4411ce
runtime.goparkunlock(...)
/usr/lib/go/src/runtime/proc.go:408
runtime.bgsweep(0xc0000380e0)
/usr/lib/go/src/runtime/mgcsweep.go:318 +0xdf fp=0xc0000797c8 sp=0xc000079780 pc=0x42c81f
runtime.gcenable.gowrap1()
/usr/lib/go/src/runtime/mgc.go:203 +0x25 fp=0xc0000797e0 sp=0xc0000797c8 pc=0x421105
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000797e8 sp=0xc0000797e0 pc=0x473ca1
created by runtime.gcenable in goroutine 1
/usr/lib/go/src/runtime/mgc.go:203 +0x66
goroutine 4 gp=0xc000007340 m=nil [GC scavenge wait]:
runtime.gopark(0x10000?, 0x3b9aca00?, 0x0?, 0x0?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000079f78 sp=0xc000079f58 pc=0x4411ce
runtime.goparkunlock(...)
/usr/lib/go/src/runtime/proc.go:408
runtime.(*scavengerState).park(0x14bcc60)
/usr/lib/go/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc000079fa8 sp=0xc000079f78 pc=0x42a1a9
runtime.bgscavenge(0xc0000380e0)
/usr/lib/go/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc000079fc8 sp=0xc000079fa8 pc=0x42a759
runtime.gcenable.gowrap2()
/usr/lib/go/src/runtime/mgc.go:204 +0x25 fp=0xc000079fe0 sp=0xc000079fc8 pc=0x4210a5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000079fe8 sp=0xc000079fe0 pc=0x473ca1
created by runtime.gcenable in goroutine 1
/usr/lib/go/src/runtime/mgc.go:204 +0xa5
goroutine 5 gp=0xc000007c00 m=nil [finalizer wait]:
runtime.gopark(0xc000078648?, 0x4144c5?, 0xa8?, 0x1?, 0xaea740?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000078620 sp=0xc000078600 pc=0x4411ce
runtime.runfinq()
/usr/lib/go/src/runtime/mfinal.go:194 +0x107 fp=0xc0000787e0 sp=0xc000078620 pc=0x420147
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000787e8 sp=0xc0000787e0 pc=0x473ca1
created by runtime.createfing in goroutine 1
/usr/lib/go/src/runtime/mfinal.go:164 +0x3d
goroutine 6 gp=0xc000398e00 m=nil [select, locked to thread]:
runtime.gopark(0xc00007a7a8?, 0x2?, 0x69?, 0x14?, 0xc00007a794?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc00007a638 sp=0xc00007a618 pc=0x4411ce
runtime.selectgo(0xc00007a7a8, 0xc00007a790, 0x0?, 0x0, 0x0?, 0x1)
/usr/lib/go/src/runtime/select.go:327 +0x725 fp=0xc00007a758 sp=0xc00007a638 pc=0x4524e5
runtime.ensureSigM.func1()
/usr/lib/go/src/runtime/signal_unix.go:1034 +0x19f fp=0xc00007a7e0 sp=0xc00007a758 pc=0x46b0ff
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00007a7e8 sp=0xc00007a7e0 pc=0x473ca1
created by runtime.ensureSigM in goroutine 1
/usr/lib/go/src/runtime/signal_unix.go:1017 +0xc8
goroutine 7 gp=0xc000398fc0 m=5 mp=0xc000100008 [syscall]:
runtime.notetsleepg(0x151f300, 0xffffffffffffffff)
/usr/lib/go/src/runtime/lock_futex.go:246 +0x29 fp=0xc00007afa0 sp=0xc00007af78 pc=0x412ae9
os/signal.signal_recv()
/usr/lib/go/src/runtime/sigqueue.go:152 +0x29 fp=0xc00007afc0 sp=0xc00007afa0 pc=0x470709
os/signal.loop()
/usr/lib/go/src/os/signal/signal_unix.go:23 +0x13 fp=0xc00007afe0 sp=0xc00007afc0 pc=0x70d753
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00007afe8 sp=0xc00007afe0 pc=0x473ca1
created by os/signal.Notify.func1.1 in goroutine 1
/usr/lib/go/src/os/signal/signal.go:151 +0x1f
goroutine 8 gp=0xc000399180 m=nil [chan receive]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc00007b718 sp=0xc00007b6f8 pc=0x4411ce
runtime.chanrecv(0xc0004ae660, 0x0, 0x1)
/usr/lib/go/src/runtime/chan.go:583 +0x3bf fp=0xc00007b790 sp=0xc00007b718 pc=0x40cd3f
runtime.chanrecv1(0x0?, 0x0?)
/usr/lib/go/src/runtime/chan.go:442 +0x12 fp=0xc00007b7b8 sp=0xc00007b790 pc=0x40c952
github.com/jmorganca/ollama/server.Serve.func1()
/home/kainoa/.local/share/ollama-build/server/routes.go:972 +0x25 fp=0xc00007b7e0 sp=0xc00007b7b8 pc=0x9b7ac5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00007b7e8 sp=0xc00007b7e0 pc=0x473ca1
created by github.com/jmorganca/ollama/server.Serve in goroutine 1
/home/kainoa/.local/share/ollama-build/server/routes.go:971 +0x458
goroutine 31 gp=0xc000399500 m=nil [IO wait]:
runtime.gopark(0xc0005e7968?, 0x41cad8?, 0x58?, 0xe2?, 0xb?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc0005e7910 sp=0xc0005e78f0 pc=0x4411ce
runtime.netpollblock(0x4851d8?, 0x409ec6?, 0x0?)
/usr/lib/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc0005e7948 sp=0xc0005e7910 pc=0x439fd7
internal/poll.runtime_pollWait(0x75cff2c165d8, 0x72)
/usr/lib/go/src/runtime/netpoll.go:345 +0x85 fp=0xc0005e7968 sp=0xc0005e7948 pc=0x46e3a5
internal/poll.(*pollDesc).wait(0xc000482880?, 0xc0005dc000?, 0x0)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0005e7990 sp=0xc0005e7968 pc=0x4f7767
internal/poll.(*pollDesc).waitRead(...)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000482880, {0xc0005dc000, 0x1000, 0x1000})
/usr/lib/go/src/internal/poll/fd_unix.go:164 +0x27a fp=0xc0005e7a28 sp=0xc0005e7990 pc=0x4f8a5a
net.(*netFD).Read(0xc000482880, {0xc0005dc000?, 0xc0005e7a98?, 0x4f7c25?})
/usr/lib/go/src/net/fd_posix.go:55 +0x25 fp=0xc0005e7a70 sp=0xc0005e7a28 pc=0x574ba5
net.(*conn).Read(0xc0005280b0, {0xc0005dc000?, 0x0?, 0xc0004a2218?})
/usr/lib/go/src/net/net.go:179 +0x45 fp=0xc0005e7ab8 sp=0xc0005e7a70 pc=0x582da5
net.(*TCPConn).Read(0xc0004a2210?, {0xc0005dc000?, 0xc000482880?, 0xc0005e7af0?})
<autogenerated>:1 +0x25 fp=0xc0005e7ae8 sp=0xc0005e7ab8 pc=0x594425
net/http.(*connReader).Read(0xc0004a2210, {0xc0005dc000, 0x1000, 0x1000})
/usr/lib/go/src/net/http/server.go:789 +0x14b fp=0xc0005e7b38 sp=0xc0005e7ae8 pc=0x6de18b
bufio.(*Reader).fill(0xc0005020c0)
/usr/lib/go/src/bufio/bufio.go:110 +0x103 fp=0xc0005e7b70 sp=0xc0005e7b38 pc=0x665243
bufio.(*Reader).Peek(0xc0005020c0, 0x4)
/usr/lib/go/src/bufio/bufio.go:148 +0x53 fp=0xc0005e7b90 sp=0xc0005e7b70 pc=0x665373
net/http.(*conn).serve(0xc00019c240, {0x108d208, 0xc0005ac690})
/usr/lib/go/src/net/http/server.go:2074 +0x749 fp=0xc0005e7fb8 sp=0xc0005e7b90 pc=0x6e3ee9
net/http.(*Server).Serve.gowrap3()
/usr/lib/go/src/net/http/server.go:3285 +0x28 fp=0xc0005e7fe0 sp=0xc0005e7fb8 pc=0x6e91e8
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0005e7fe8 sp=0xc0005e7fe0 pc=0x473ca1
created by net/http.(*Server).Serve in goroutine 1
/usr/lib/go/src/net/http/server.go:3285 +0x4b4
goroutine 11 gp=0xc0003996c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000074f50 sp=0xc000074f30 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000074fe0 sp=0xc000074f50 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000074fe8 sp=0xc000074fe0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 12 gp=0xc000399c00 m=nil [GC worker (idle)]:
runtime.gopark(0x1dd07dfc2b8?, 0x3?, 0xc0?, 0x71?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000075750 sp=0xc000075730 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000757e0 sp=0xc000075750 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000757e8 sp=0xc0000757e0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 13 gp=0xc000399dc0 m=nil [GC worker (idle)]:
runtime.gopark(0x1dd07df8b18?, 0x3?, 0xa8?, 0x79?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000075f50 sp=0xc000075f30 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000075fe0 sp=0xc000075f50 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000075fe8 sp=0xc000075fe0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 18 gp=0xc000102a80 m=nil [GC worker (idle)]:
runtime.gopark(0x1dd07dfc196?, 0x1?, 0xd2?, 0x14?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000126750 sp=0xc000126730 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0001267e0 sp=0xc000126750 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0001267e8 sp=0xc0001267e0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 19 gp=0xc000102c40 m=nil [GC worker (idle)]:
runtime.gopark(0x1dd07dfae04?, 0x3?, 0xb0?, 0x4?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000126f50 sp=0xc000126f30 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000126fe0 sp=0xc000126f50 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000126fe8 sp=0xc000126fe0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 20 gp=0xc000102e00 m=nil [GC worker (idle)]:
runtime.gopark(0x1dd07dfc8c6?, 0x3?, 0x14?, 0x23?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000127750 sp=0xc000127730 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0001277e0 sp=0xc000127750 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0001277e8 sp=0xc0001277e0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 21 gp=0xc000102fc0 m=nil [GC worker (idle)]:
runtime.gopark(0x1dd07dfc8b2?, 0x3?, 0x72?, 0xdc?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000127f50 sp=0xc000127f30 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000127fe0 sp=0xc000127f50 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000127fe8 sp=0xc000127fe0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 22 gp=0xc000103180 m=nil [GC worker (idle)]:
runtime.gopark(0x15205a0?, 0x1?, 0xc2?, 0x29?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000128750 sp=0xc000128730 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0001287e0 sp=0xc000128750 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0001287e8 sp=0xc0001287e0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 23 gp=0xc000103340 m=nil [GC worker (idle)]:
runtime.gopark(0x1dd07dfafbc?, 0x1?, 0x60?, 0x93?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000128f50 sp=0xc000128f30 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000128fe0 sp=0xc000128f50 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000128fe8 sp=0xc000128fe0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 24 gp=0xc000103500 m=nil [GC worker (idle)]:
runtime.gopark(0x15205a0?, 0x1?, 0xaf?, 0xb?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000129750 sp=0xc000129730 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0001297e0 sp=0xc000129750 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0001297e8 sp=0xc0001297e0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 25 gp=0xc0001036c0 m=nil [GC worker (idle)]:
runtime.gopark(0x1dd07df9914?, 0x1?, 0xc?, 0x89?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000129f50 sp=0xc000129f30 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000129fe0 sp=0xc000129f50 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000129fe8 sp=0xc000129fe0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 26 gp=0xc000103880 m=nil [GC worker (idle)]:
runtime.gopark(0x1dd07dfc236?, 0x3?, 0x1c?, 0xf7?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000122750 sp=0xc000122730 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0001227e0 sp=0xc000122750 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0001227e8 sp=0xc0001227e0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 27 gp=0xc000103a40 m=nil [GC worker (idle)]:
runtime.gopark(0x1dd07dfae4a?, 0x3?, 0x6c?, 0x7?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000122f50 sp=0xc000122f30 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000122fe0 sp=0xc000122f50 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000122fe8 sp=0xc000122fe0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 28 gp=0xc000103c00 m=nil [GC worker (idle)]:
runtime.gopark(0x1dd07dfaf9e?, 0x1?, 0xa4?, 0x38?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000123750 sp=0xc000123730 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0001237e0 sp=0xc000123750 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0001237e8 sp=0xc0001237e0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 29 gp=0xc000103dc0 m=nil [GC worker (idle)]:
runtime.gopark(0x1dd07df991e?, 0x3?, 0x6a?, 0x7c?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000123f50 sp=0xc000123f30 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000123fe0 sp=0xc000123f50 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000123fe8 sp=0xc000123fe0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 30 gp=0xc00012c000 m=nil [GC worker (idle)]:
runtime.gopark(0x1dd07dff864?, 0x3?, 0x2?, 0x30?, 0x0?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000124750 sp=0xc000124730 pc=0x4411ce
runtime.gcBgMarkWorker()
/usr/lib/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0001247e0 sp=0xc000124750 pc=0x4231e5
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0001247e8 sp=0xc0001247e0 pc=0x473ca1
created by runtime.gcBgMarkStartWorkers in goroutine 1
/usr/lib/go/src/runtime/mgc.go:1234 +0x1c
goroutine 16 gp=0xc000582540 m=nil [IO wait]:
runtime.gopark(0x10?, 0x10?, 0xf0?, 0x55?, 0xb?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc0001255a8 sp=0xc000125588 pc=0x4411ce
runtime.netpollblock(0x4851d8?, 0x409ec6?, 0x0?)
/usr/lib/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc0001255e0 sp=0xc0001255a8 pc=0x439fd7
internal/poll.runtime_pollWait(0x75cff2c163e8, 0x72)
/usr/lib/go/src/runtime/netpoll.go:345 +0x85 fp=0xc000125600 sp=0xc0001255e0 pc=0x46e3a5
internal/poll.(*pollDesc).wait(0xc000482080?, 0xc0003562b1?, 0x0)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc000125628 sp=0xc000125600 pc=0x4f7767
internal/poll.(*pollDesc).waitRead(...)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000482080, {0xc0003562b1, 0x1, 0x1})
/usr/lib/go/src/internal/poll/fd_unix.go:164 +0x27a fp=0xc0001256c0 sp=0xc000125628 pc=0x4f8a5a
net.(*netFD).Read(0xc000482080, {0xc0003562b1?, 0xc000125748?, 0x470410?})
/usr/lib/go/src/net/fd_posix.go:55 +0x25 fp=0xc000125708 sp=0xc0001256c0 pc=0x574ba5
net.(*conn).Read(0xc000528000, {0xc0003562b1?, 0x0?, 0x151e740?})
/usr/lib/go/src/net/net.go:179 +0x45 fp=0xc000125750 sp=0xc000125708 pc=0x582da5
net.(*TCPConn).Read(0x14551d0?, {0xc0003562b1?, 0x0?, 0x0?})
<autogenerated>:1 +0x25 fp=0xc000125780 sp=0xc000125750 pc=0x594425
net/http.(*connReader).backgroundRead(0xc0003562a0)
/usr/lib/go/src/net/http/server.go:681 +0x37 fp=0xc0001257c8 sp=0xc000125780 pc=0x6ddcf7
net/http.(*connReader).startBackgroundRead.gowrap2()
/usr/lib/go/src/net/http/server.go:677 +0x25 fp=0xc0001257e0 sp=0xc0001257c8 pc=0x6ddc25
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0001257e8 sp=0xc0001257e0 pc=0x473ca1
created by net/http.(*connReader).startBackgroundRead in goroutine 32
/usr/lib/go/src/net/http/server.go:677 +0xba
goroutine 37 gp=0xc000582700 m=nil [IO wait]:
runtime.gopark(0x430?, 0xc000351958?, 0x40?, 0x19?, 0xb?)
/usr/lib/go/src/runtime/proc.go:402 +0xce fp=0xc000351910 sp=0xc0003518f0 pc=0x4411ce
runtime.netpollblock(0x4851d8?, 0x409ec6?, 0x0?)
/usr/lib/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc000351948 sp=0xc000351910 pc=0x439fd7
internal/poll.runtime_pollWait(0x75cff2c164e0, 0x72)
/usr/lib/go/src/runtime/netpoll.go:345 +0x85 fp=0xc000351968 sp=0xc000351948 pc=0x46e3a5
internal/poll.(*pollDesc).wait(0xc000048100?, 0xc0006c4000?, 0x0)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc000351990 sp=0xc000351968 pc=0x4f7767
internal/poll.(*pollDesc).waitRead(...)
/usr/lib/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000048100, {0xc0006c4000, 0x1000, 0x1000})
/usr/lib/go/src/internal/poll/fd_unix.go:164 +0x27a fp=0xc000351a28 sp=0xc000351990 pc=0x4f8a5a
net.(*netFD).Read(0xc000048100, {0xc0006c4000?, 0xc000351a98?, 0x4f7c25?})
/usr/lib/go/src/net/fd_posix.go:55 +0x25 fp=0xc000351a70 sp=0xc000351a28 pc=0x574ba5
net.(*conn).Read(0xc00007c000, {0xc0006c4000?, 0x0?, 0xc0000ba4e8?})
/usr/lib/go/src/net/net.go:179 +0x45 fp=0xc000351ab8 sp=0xc000351a70 pc=0x582da5
net.(*TCPConn).Read(0xc0000ba4e0?, {0xc0006c4000?, 0xc000048100?, 0xc000351af0?})
<autogenerated>:1 +0x25 fp=0xc000351ae8 sp=0xc000351ab8 pc=0x594425
net/http.(*connReader).Read(0xc0000ba4e0, {0xc0006c4000, 0x1000, 0x1000})
/usr/lib/go/src/net/http/server.go:789 +0x14b fp=0xc000351b38 sp=0xc000351ae8 pc=0x6de18b
bufio.(*Reader).fill(0xc0001160c0)
/usr/lib/go/src/bufio/bufio.go:110 +0x103 fp=0xc000351b70 sp=0xc000351b38 pc=0x665243
bufio.(*Reader).Peek(0xc0001160c0, 0x4)
/usr/lib/go/src/bufio/bufio.go:148 +0x53 fp=0xc000351b90 sp=0xc000351b70 pc=0x665373
net/http.(*conn).serve(0xc0005de120, {0x108d208, 0xc0005ac690})
/usr/lib/go/src/net/http/server.go:2074 +0x749 fp=0xc000351fb8 sp=0xc000351b90 pc=0x6e3ee9
net/http.(*Server).Serve.gowrap3()
/usr/lib/go/src/net/http/server.go:3285 +0x28 fp=0xc000351fe0 sp=0xc000351fb8 pc=0x6e91e8
runtime.goexit({})
/usr/lib/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000351fe8 sp=0xc000351fe0 pc=0x473ca1
created by net/http.(*Server).Serve in goroutine 1
/usr/lib/go/src/net/http/server.go:3285 +0x4b4
rax 0x0
rbx 0x75ccbe831690
rcx 0x75cf60d00080
rdx 0x1c0
rdi 0x75cf60d00080
rsi 0x75ccbece3690
rbp 0x75cf9b3fe9d0
rsp 0x75cf9b3fe7a0
r8 0x90
r9 0x4
r10 0x1
r11 0x1
r12 0x15
r13 0x75cf4490ef40
r14 0x0
r15 0x75cf9b3fead0
rip 0x75cf8ab2b380
rflags 0x10246
cs 0x33
fs 0x0
gs 0x0
@ThatOneCalculator thanks for the updated log. Based on the output, it looks like the segfault is in llama.cpp or rocm code. It looks potentially similar to https://github.com/ggerganov/llama.cpp/issues/4939 which we'll keep an eye on.
As opposed to the others here, those changes seem to have fixed the crash I was experiencing. I'll update if further investigation proves otherwise.
Just fixed it!! Here's what I did:
- Uninstall all
rocm-*packages - Install
opencl-amd-dev,amdgpu-pro-oglp, andllm-clblast-git - Reboot
-
cd /opt && sudo ln -s rocm-6.0.0 rocm - Do a fresh
git cloneand build with:
CLBlast_DIR=/usr/lib/cmake/CLBlast AMDGPU_TARGETS="gfx1030" HSA_OVERRIDE_GFX_VERSION=10.3.0 ROCM_PATH=/opt/rocm go generate -tags rocm ./... && go build -tags rocm && sudo cp ./ollama /usr/bin/ollama
- Serve with
env GIN_MODE=release HCC_AMDGPU_TARGET=gfx1030 OLLAMA_ORIGINS="*" HSA_OVERRIDE_GFX_VERSION=10.3.0 ROCM_PATH=/opt/rocm-6.0.0 OLLAMA_DEBUG=1 ollama serve
(probably) related packages I have installed:
❯ yay -Q | grep "opencl"
opencl-amd 1:6.0.0-1
opencl-amd-dev 1:6.0.0-2
opencl-clover-mesa 1:23.3.4-3
opencl-headers 2:2023.04.17-2
opencl-rusticl-mesa 1:23.3.4-3
❯ yay -Q | grep "clblast"
clblast-git 1.6.1.8.g162783a4-1
❯ yay -Q | grep "amdgpu-pro"
amdgpu-pro-oglp 23.40_1710631-1
amf-amdgpu-pro 23.30_1697785-1
vulkan-amdgpu-pro 23.30_1697785-1