LocalAI
LocalAI copied to clipboard
mudler
unexpectedly reached end of fileSIGILL: illegal instruction
I'm trying to use many models, but none of them are working when I use the /chat/completions page. Here's what I'm getting with gpt4all:
I'm using Postman with this query:
{ "model": "gpt", "messages": [ { "role": "user", "content": "How are you?" } ], "temperature": 0.9 }
Starting LocalAI using 4 threads, with models path: /models ┌───────────────────────────────────────────────────┐ │ Fiber v2.45.0 │ │ http://127.0.0.1:8080 │ │ (bound on host 0.0.0.0 and port 8080) │ │ │ │ Handlers ............ 17 Processes ........... 1 │ │ Prefork ....... Disabled PID ............... 164 │ └───────────────────────────────────────────────────┘ llama.cpp: loading model from /models/gpt failed unexpectedly reached end of fileSIGILL: illegal instruction PC=0xa893c4 m=5 sigcode=2 signal arrived during cgo execution instruction bytes: 0xc4 0xc3 0x7d 0x39 0x86 0xd8 0x13 0x0 0x0 0x1 0x49 0x89 0x86 0xb8 0x14 0x0 goroutine 18 [syscall]: runtime.cgocall(0x97b760, 0xc0001ae920) /usr/local/go/src/runtime/cgocall.go:157 +0x5c fp=0xc0001ae8f8 sp=0xc0001ae8c0 pc=0x44a4fc github.com/nomic-ai/gpt4all/gpt4all-bindings/golang._Cfunc_load_gptjllama_model(0x7fa4e8002550, 0x4) cgo_gotypes.go:137 +0x4d fp=0xc0001ae920 sp=0xc0001ae8f8 pc=0x584f0d github.com/nomic-ai/gpt4all/gpt4all-bindings/golang.New({0xc000236650, 0xb}, {0xc000209080, 0x2, 0x1?}) /build/gpt4all/gpt4all-bindings/golang/gpt4all.go:35 +0x145 fp=0xc0001ae978 sp=0xc0001ae920 pc=0x585225 github.com/go-skynet/LocalAI/pkg/model.gpt4allLM.func1({0xc000236650?, 0xbca30f?}) /build/pkg/model/initializers.go:110 +0x2a fp=0xc0001ae9b0 sp=0xc0001ae978 pc=0x607d4a github.com/go-skynet/LocalAI/pkg/model.(*ModelLoader).LoadModel(0xc0001a66c0, {0xc000236568, 0x3}, 0xc000024ca0) /build/pkg/model/loader.go:127 +0x1fe fp=0xc0001aeaa8 sp=0xc0001ae9b0 pc=0x609c9e github.com/go-skynet/LocalAI/pkg/model.(*ModelLoader).BackendLoader(0xc0001a66c0, {0xbba207, 0xd}, {0xc000236568, 0x3}, {0xc000014590, 0x1, 0x1}, 0x4) /build/pkg/model/initializers.go:150 +0x7d2 fp=0xc0001aeb70 sp=0xc0001aeaa8 pc=0x608752 github.com/go-skynet/LocalAI/pkg/model.(*ModelLoader).GreedyLoader(0xc0001a66c0, {0xc000236568, 0x3}, {0xc000014590, 0x1, 0x1}, 0x0?) /build/pkg/model/initializers.go:183 +0x2e5 fp=0xc0001aeca8 sp=0xc0001aeb70 pc=0x608ca5 github.com/go-skynet/LocalAI/api.ModelInference({, _}, _, {{{0xc000236568, 0x3}, {0x0, 0x0}, {0x0, 0x0}, {0x0, ...}, ...}, ...}, ...) /build/api/prediction.go:218 +0x145 fp=0xc0001aef58 sp=0xc0001aeca8 pc=0x8d8325 github.com/go-skynet/LocalAI/api.ComputeChoices({0xc0002400c0, 0xba}, 0xc00014ab40, 0xc000115b80, 0xc000208f50?, 0xbf46b8, 0x4?) /build/api/prediction.go:517 +0x138 fp=0xc0001af708 sp=0xc0001aef58 pc=0x8db9f8 github.com/go-skynet/LocalAI/api.chatEndpoint.func2(0xc000132840) /build/api/openai.go:361 +0x8ec fp=0xc0001af8c8 sp=0xc0001af708 pc=0x8d35cc github.com/gofiber/fiber/v2.(*App).next(0xc000138d80, 0xc000132840) /go/pkg/mod/github.com/gofiber/fiber/[email protected]/router.go:144 +0x1bf fp=0xc0001af970 sp=0xc0001af8c8 pc=0x88993f github.com/gofiber/fiber/v2.(*Ctx).Next(0xc0000a0330?) /go/pkg/mod/github.com/gofiber/fiber/[email protected]/ctx.go:913 +0x53 fp=0xc0001af990 sp=0xc0001af970 pc=0x874f13 github.com/gofiber/fiber/v2/middleware/cors.New.func1(0xc000132840) /go/pkg/mod/github.com/gofiber/fiber/[email protected]/middleware/cors/cors.go:162 +0x3da fp=0xc0001afa98 sp=0xc0001af990 pc=0x88f75a github.com/gofiber/fiber/v2.(*Ctx).Next(0x14?) /go/pkg/mod/github.com/gofiber/fiber/[email protected]/ctx.go:910 +0x43 fp=0xc0001afab8 sp=0xc0001afa98 pc=0x874f03 github.com/gofiber/fiber/v2/middleware/recover.New.func1(0xb3c360?) /go/pkg/mod/github.com/gofiber/fiber/[email protected]/middleware/recover/recover.go:43 +0xcb fp=0xc0001afb30 sp=0xc0001afab8 pc=0x89638b github.com/gofiber/fiber/v2.(*App).next(0xc000138d80, 0xc000132840) /go/pkg/mod/github.com/gofiber/fiber/[email protected]/router.go:144 +0x1bf fp=0xc0001afbd8 sp=0xc0001afb30 pc=0x88993f github.com/gofiber/fiber/v2.(*App).handler(0xc000138d80, 0x50ce57?) /go/pkg/mod/github.com/gofiber/fiber/[email protected]/router.go:171 +0x87 fp=0xc0001afc38 sp=0xc0001afbd8 pc=0x889b87 github.com/gofiber/fiber/v2.(*App).handler-fm(0xc0000a0000?)
:1 +0x2c fp=0xc0001afc58 sp=0xc0001afc38 pc=0x88edac github.com/valyala/fasthttp.(*Server).serveConn(0xc0001fc400, {0xc79c80?, 0xc00008e008}) /go/pkg/mod/github.com/valyala/[email protected]/server.go:2365 +0x11d3 fp=0xc0001afec8 sp=0xc0001afc58 pc=0x80fb33 github.com/valyala/fasthttp.(*Server).serveConn-fm({0xc79c80?, 0xc00008e008?}) :1 +0x39 fp=0xc0001afef0 sp=0xc0001afec8 pc=0x81f3f9 github.com/valyala/fasthttp.(*workerPool).workerFunc(0xc000117680, 0xc00009e000) /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:224 +0xa9 fp=0xc0001affa0 sp=0xc0001afef0 pc=0x81b629 github.com/valyala/fasthttp.(*workerPool).getCh.func1() /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:196 +0x38 fp=0xc0001affe0 sp=0xc0001affa0 pc=0x81b398 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0001affe8 sp=0xc0001affe0 pc=0x4ac9c1 created by github.com/valyala/fasthttp.(*workerPool).getCh /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:195 +0x1b0 goroutine 1 [IO wait]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc0001cd418 sp=0xc0001cd3f8 pc=0x47e016 runtime.netpollblock(0x7fa500316548?, 0x449b8f?, 0x0?) /usr/local/go/src/runtime/netpoll.go:527 +0xf7 fp=0xc0001cd450 sp=0xc0001cd418 pc=0x476977 internal/poll.runtime_pollWait(0x7fa5002ea418, 0x72) /usr/local/go/src/runtime/netpoll.go:306 +0x89 fp=0xc0001cd470 sp=0xc0001cd450 pc=0x4a76c9 internal/poll.(*pollDesc).wait(0xc000166b00?, 0x4?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc0001cd498 sp=0xc0001cd470 pc=0x51ae92 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Accept(0xc000166b00) /usr/local/go/src/internal/poll/fd_unix.go:614 +0x2bd fp=0xc0001cd540 sp=0xc0001cd498 pc=0x52079d net.(*netFD).accept(0xc000166b00) /usr/local/go/src/net/fd_unix.go:172 +0x35 fp=0xc0001cd5f8 sp=0xc0001cd540 pc=0x5a8c15 net.(*TCPListener).accept(0xc000012828) /usr/local/go/src/net/tcpsock_posix.go:148 +0x25 fp=0xc0001cd620 sp=0xc0001cd5f8 pc=0x5beea5 net.(*TCPListener).Accept(0xc000012828) /usr/local/go/src/net/tcpsock.go:297 +0x3d fp=0xc0001cd650 sp=0xc0001cd620 pc=0x5bdf9d github.com/valyala/fasthttp.acceptConn(0xc0001fc400, {0xc773e0, 0xc000012828}, 0xc0001cd848) /go/pkg/mod/github.com/valyala/[email protected]/server.go:1930 +0x62 fp=0xc0001cd730 sp=0xc0001cd650 pc=0x80e002 github.com/valyala/fasthttp.(*Server).Serve(0xc0001fc400, {0xc773e0?, 0xc000012828}) /go/pkg/mod/github.com/valyala/[email protected]/server.go:1823 +0x4f4 fp=0xc0001cd878 sp=0xc0001cd730 pc=0x80d614 github.com/gofiber/fiber/v2.(*App).Listen(0xc000138d80, {0xbb0b88?, 0x7?}) /go/pkg/mod/github.com/gofiber/fiber/[email protected]/listen.go:82 +0x110 fp=0xc0001cd8d8 sp=0xc0001cd878 pc=0x880a30 main.main.func1(0xc0001cdbc8?) /build/main.go:96 +0x2f9 fp=0xc0001cd9b8 sp=0xc0001cd8d8 pc=0x909919 github.com/urfave/cli/v2.(*Command).Run(0xc0001ba160, 0xc00006a800, {0xc000024060, 0x2, 0x2}) /go/pkg/mod/github.com/urfave/cli/[email protected]/command.go:274 +0x9eb fp=0xc0001cdc58 sp=0xc0001cd9b8 pc=0x8f786b github.com/urfave/cli/v2.(*App).RunContext(0xc0001b6000, {0xc776b8?, 0xc000028078}, {0xc000024060, 0x2, 0x2}) /go/pkg/mod/github.com/urfave/cli/[email protected]/app.go:332 +0x616 fp=0xc0001cdcc8 sp=0xc0001cdc58 pc=0x8f4676 github.com/urfave/cli/v2.(*App).Run(...) /go/pkg/mod/github.com/urfave/cli/[email protected]/app.go:309 main.main() /build/main.go:100 +0xbae fp=0xc0001cdf80 sp=0xc0001cdcc8 pc=0x90954e runtime.main() /usr/local/go/src/runtime/proc.go:250 +0x207 fp=0xc0001cdfe0 sp=0xc0001cdf80 pc=0x47dbe7 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0001cdfe8 sp=0xc0001cdfe0 pc=0x4ac9c1 goroutine 2 [force gc (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000040fb0 sp=0xc000040f90 pc=0x47e016 runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:387 runtime.forcegchelper() /usr/local/go/src/runtime/proc.go:305 +0xb0 fp=0xc000040fe0 sp=0xc000040fb0 pc=0x47de50 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000040fe8 sp=0xc000040fe0 pc=0x4ac9c1 created by runtime.init.6 /usr/local/go/src/runtime/proc.go:293 +0x25 goroutine 3 [GC sweep wait]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000041780 sp=0xc000041760 pc=0x47e016 runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:387 runtime.bgsweep(0x0?) /usr/local/go/src/runtime/mgcsweep.go:278 +0x8e fp=0xc0000417c8 sp=0xc000041780 pc=0x46a22e runtime.gcenable.func1() /usr/local/go/src/runtime/mgc.go:178 +0x26 fp=0xc0000417e0 sp=0xc0000417c8 pc=0x45f4e6 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000417e8 sp=0xc0000417e0 pc=0x4ac9c1 created by runtime.gcenable /usr/local/go/src/runtime/mgc.go:178 +0x6b goroutine 4 [GC scavenge wait]: runtime.gopark(0xc000068000?, 0xc701e8?, 0x1?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000041f70 sp=0xc000041f50 pc=0x47e016 runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:387 runtime.(*scavengerState).park(0x1060e60) /usr/local/go/src/runtime/mgcscavenge.go:400 +0x53 fp=0xc000041fa0 sp=0xc000041f70 pc=0x468153 runtime.bgscavenge(0x0?) /usr/local/go/src/runtime/mgcscavenge.go:628 +0x45 fp=0xc000041fc8 sp=0xc000041fa0 pc=0x468725 runtime.gcenable.func2() /usr/local/go/src/runtime/mgc.go:179 +0x26 fp=0xc000041fe0 sp=0xc000041fc8 pc=0x45f486 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000041fe8 sp=0xc000041fe0 pc=0x4ac9c1 created by runtime.gcenable /usr/local/go/src/runtime/mgc.go:179 +0xaa goroutine 5 [finalizer wait]: runtime.gopark(0x1a0?, 0x1061b40?, 0x60?, 0x78?, 0xc000040770?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000040628 sp=0xc000040608 pc=0x47e016 runtime.runfinq() /usr/local/go/src/runtime/mfinal.go:193 +0x107 fp=0xc0000407e0 sp=0xc000040628 pc=0x45e527 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000407e8 sp=0xc0000407e0 pc=0x4ac9c1 created by runtime.createfing /usr/local/go/src/runtime/mfinal.go:163 +0x45 goroutine 6 [sleep]: runtime.gopark(0xf34e8098faa?, 0xc000042788?, 0x5?, 0xd8?, 0xc0001176b0?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000042758 sp=0xc000042738 pc=0x47e016 time.Sleep(0x2540be400) /usr/local/go/src/runtime/time.go:195 +0x135 fp=0xc000042798 sp=0xc000042758 pc=0x4a9835 github.com/valyala/fasthttp.(*workerPool).Start.func2() /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:67 +0x56 fp=0xc0000427e0 sp=0xc000042798 pc=0x81aaf6 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000427e8 sp=0xc0000427e0 pc=0x4ac9c1 created by github.com/valyala/fasthttp.(*workerPool).Start /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:59 +0xdd rax 0x7fa4e8005398 rbx 0xffffffff rcx 0x20 rdx 0x4c46d8c rdi 0x7fa4e8005408 rsi 0x16e0 rbp 0x7fa4fbffebf0 rsp 0x7fa4fbffd730 r8 0x7fa4e8003eb0 r9 0x7fa4e8000080 r10 0x4 r11 0x0 r12 0x7fa4fbffec70 r13 0x7fa4fbffedb0 r14 0x7fa4e8003eb0 r15 0xc000080000 rip 0xa893c4 rflags 0x10246 cs 0x33 fs 0x0 gs 0x0
Hi, @CaioLuppo Can you try other models from https://github.com/go-skynet/model-gallery/blob/3d2c9e259f6b51aa618b42a9f891521bbd253f79/gpt4all-j.yaml#LL22C20-L22C52. And please share more detail about your environment like:
-
localAIversion - model address(if you can)
Which way do you run localAI?
Hi, Aisuko! All I've done so far is take the README.md file and try to run the "Use chatgpt4all" example. I'm using LocalAI 1.13.0, the latest in this repository via docker, as far as I know.
I've just tried the gpt model you suggested, but nearly the same error happened:
SIGILL: illegal instruction PC=0xad522d m=3 sigcode=2 signal arrived during cgo execution instruction bytes: 0xc4 0xe2 0x71 0xa9 0x15 0xe 0xd7 0x25 0x0 0xc4 0x81 0x7a 0x11 0x4c 0xbd 0x0 goroutine 21 [syscall]: runtime.cgocall(0x9e7fe0, 0xc000166a58) /usr/local/go/src/runtime/cgocall.go:157 +0x5c fp=0xc000166a30 sp=0xc0001669f8 pc=0x44a59c github.com/nomic-ai/gpt4all/gpt4all-bindings/golang._Cfunc_load_gptj_model(0x7f1fa0000ca0, 0x4) cgo_gotypes.go:123 +0x4d fp=0xc000166a58 sp=0xc000166a30 pc=0x58890d github.com/nomic-ai/gpt4all/gpt4all-bindings/golang.New({0xc0000d45b8, 0x16}, {0xc0001c5950, 0x2, 0x1?}) /build/gpt4all/gpt4all-bindings/golang/gpt4all.go:37 +0x10c fp=0xc000166ab0 sp=0xc000166a58 pc=0x588cac github.com/go-skynet/LocalAI/pkg/model.gpt4allLM.func1({0xc0000d45b8?, 0xc4edff?}) /build/pkg/model/initializers.go:110 +0x2a fp=0xc000166ae8 sp=0xc000166ab0 pc=0x608aaa github.com/go-skynet/LocalAI/pkg/model.(*ModelLoader).LoadModel(0xc00015e9f0, {0xc0001d0f70, 0xe}, 0xc0000aede0) /build/pkg/model/loader.go:127 +0x1fe fp=0xc000166be0 sp=0xc000166ae8 pc=0x60aabe github.com/go-skynet/LocalAI/pkg/model.(*ModelLoader).BackendLoader(0xc00015e9f0, {0xc0001d0ef7, 0x9}, {0xc0001d0f70, 0xe}, {0xc0000aa618, 0x1, 0x1}, 0x4) /build/pkg/model/initializers.go:154 +0x53c fp=0xc000166ca8 sp=0xc000166be0 pc=0x60921c github.com/go-skynet/LocalAI/api.ModelInference({, _}, _, {{{0xc0001d0f70, 0xe}, {0x0, 0x0}, {0x0, 0x0}, {0x0, ...}, ...}, ...}, ...) /build/api/prediction.go:220 +0x17d fp=0xc000166f58 sp=0xc000166ca8 pc=0x944a7d github.com/go-skynet/LocalAI/api.ComputeChoices({0xc0002860c0, 0xb5}, 0xc0000f4dc0, 0xc0001ca780, 0xc0001c58b0?, 0xc7a300, 0x4?) /build/api/prediction.go:517 +0x138 fp=0xc000167708 sp=0xc000166f58 pc=0x948118 github.com/go-skynet/LocalAI/api.chatEndpoint.func2(0xc0000ddb80) /build/api/openai.go:361 +0x8ec fp=0xc0001678c8 sp=0xc000167708 pc=0x93fd0c github.com/gofiber/fiber/v2.(*App).next(0xc0000e1200, 0xc0000ddb80) /go/pkg/mod/github.com/gofiber/fiber/[email protected]/router.go:144 +0x1bf fp=0xc000167970 sp=0xc0001678c8 pc=0x8c4e5f github.com/gofiber/fiber/v2.(*Ctx).Next(0xc0001f8330?) /go/pkg/mod/github.com/gofiber/fiber/[email protected]/ctx.go:913 +0x53 fp=0xc000167990 sp=0xc000167970 pc=0x8b0433 github.com/gofiber/fiber/v2/middleware/cors.New.func1(0xc0000ddb80) /go/pkg/mod/github.com/gofiber/fiber/[email protected]/middleware/cors/cors.go:162 +0x3da fp=0xc000167a98 sp=0xc000167990 pc=0x8cac7a github.com/gofiber/fiber/v2.(*Ctx).Next(0x14?) /go/pkg/mod/github.com/gofiber/fiber/[email protected]/ctx.go:910 +0x43 fp=0xc000167ab8 sp=0xc000167a98 pc=0x8b0423 github.com/gofiber/fiber/v2/middleware/recover.New.func1(0xbb29a0?) /go/pkg/mod/github.com/gofiber/fiber/[email protected]/middleware/recover/recover.go:43 +0xcb fp=0xc000167b30 sp=0xc000167ab8 pc=0x8d18ab github.com/gofiber/fiber/v2.(*App).next(0xc0000e1200, 0xc0000ddb80) /go/pkg/mod/github.com/gofiber/fiber/[email protected]/router.go:144 +0x1bf fp=0xc000167bd8 sp=0xc000167b30 pc=0x8c4e5f github.com/gofiber/fiber/v2.(*App).handler(0xc0000e1200, 0x4cf3b7?) /go/pkg/mod/github.com/gofiber/fiber/[email protected]/router.go:171 +0x87 fp=0xc000167c38 sp=0xc000167bd8 pc=0x8c50a7 github.com/gofiber/fiber/v2.(*App).handler-fm(0xc0001f8000?)
:1 +0x2c fp=0xc000167c58 sp=0xc000167c38 pc=0x8ca2cc github.com/valyala/fasthttp.(*Server).serveConn(0xc0001ae400, {0xd0b240?, 0xc0000aa538}) /go/pkg/mod/github.com/valyala/[email protected]/server.go:2365 +0x11d3 fp=0xc000167ec8 sp=0xc000167c58 pc=0x84b053 github.com/valyala/fasthttp.(*Server).serveConn-fm({0xd0b240?, 0xc0000aa538?}) :1 +0x39 fp=0xc000167ef0 sp=0xc000167ec8 pc=0x85a919 github.com/valyala/fasthttp.(*workerPool).workerFunc(0xc0001d2820, 0xc0000aece0) /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:224 +0xa9 fp=0xc000167fa0 sp=0xc000167ef0 pc=0x856b49 github.com/valyala/fasthttp.(*workerPool).getCh.func1() /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:196 +0x38 fp=0xc000167fe0 sp=0xc000167fa0 pc=0x8568b8 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000167fe8 sp=0xc000167fe0 pc=0x4ad1a1 created by github.com/valyala/fasthttp.(*workerPool).getCh /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:195 +0x1b0 goroutine 1 [IO wait]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc00018f3f8 sp=0xc00018f3d8 pc=0x47e396 runtime.netpollblock(0x7f1fd47e49e8?, 0x449c2f?, 0x0?) /usr/local/go/src/runtime/netpoll.go:527 +0xf7 fp=0xc00018f430 sp=0xc00018f3f8 pc=0x476cf7 internal/poll.runtime_pollWait(0x7f1fac468b58, 0x72) /usr/local/go/src/runtime/netpoll.go:306 +0x89 fp=0xc00018f450 sp=0xc00018f430 pc=0x4a7a49 internal/poll.(*pollDesc).wait(0xc00011ed80?, 0x4?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc00018f478 sp=0xc00018f450 pc=0x51e7b2 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Accept(0xc00011ed80) /usr/local/go/src/internal/poll/fd_unix.go:614 +0x2bd fp=0xc00018f520 sp=0xc00018f478 pc=0x5240bd net.(*netFD).accept(0xc00011ed80) /usr/local/go/src/net/fd_unix.go:172 +0x35 fp=0xc00018f5d8 sp=0xc00018f520 pc=0x5a9855 net.(*TCPListener).accept(0xc0000a8828) /usr/local/go/src/net/tcpsock_posix.go:148 +0x25 fp=0xc00018f600 sp=0xc00018f5d8 pc=0x5bfc05 net.(*TCPListener).Accept(0xc0000a8828) /usr/local/go/src/net/tcpsock.go:297 +0x3d fp=0xc00018f630 sp=0xc00018f600 pc=0x5becfd github.com/valyala/fasthttp.acceptConn(0xc0001ae400, {0xd08860, 0xc0000a8828}, 0xc00018f828) /go/pkg/mod/github.com/valyala/[email protected]/server.go:1930 +0x62 fp=0xc00018f710 sp=0xc00018f630 pc=0x849522 github.com/valyala/fasthttp.(*Server).Serve(0xc0001ae400, {0xd08860?, 0xc0000a8828}) /go/pkg/mod/github.com/valyala/[email protected]/server.go:1823 +0x4f4 fp=0xc00018f858 sp=0xc00018f710 pc=0x848b34 github.com/gofiber/fiber/v2.(*App).Listen(0xc0000e1200, {0xc34934?, 0x7?}) /go/pkg/mod/github.com/gofiber/fiber/[email protected]/listen.go:82 +0x110 fp=0xc00018f8b8 sp=0xc00018f858 pc=0x8bbf50 main.main.func1(0xc00018fbc8?) /build/main.go:97 +0x345 fp=0xc00018f9b8 sp=0xc00018f8b8 pc=0x9761c5 github.com/urfave/cli/v2.(*Command).Run(0xc0001b8160, 0xc0000e2900, {0xc0000ae000, 0x2, 0x2}) /go/pkg/mod/github.com/urfave/cli/[email protected]/command.go:274 +0x9eb fp=0xc00018fc58 sp=0xc00018f9b8 pc=0x9640cb github.com/urfave/cli/v2.(*App).RunContext(0xc0001b4000, {0xd08bc8?, 0xc0000a0000}, {0xc0000ae000, 0x2, 0x2}) /go/pkg/mod/github.com/urfave/cli/[email protected]/app.go:332 +0x616 fp=0xc00018fcc8 sp=0xc00018fc58 pc=0x960ed6 github.com/urfave/cli/v2.(*App).Run(...) /go/pkg/mod/github.com/urfave/cli/[email protected]/app.go:309 main.main() /build/main.go:101 +0xbae fp=0xc00018ff80 sp=0xc00018fcc8 pc=0x975dae runtime.main() /usr/local/go/src/runtime/proc.go:250 +0x207 fp=0xc00018ffe0 sp=0xc00018ff80 pc=0x47df67 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00018ffe8 sp=0xc00018ffe0 pc=0x4ad1a1 goroutine 2 [force gc (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000040fb0 sp=0xc000040f90 pc=0x47e396 runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:387 runtime.forcegchelper() /usr/local/go/src/runtime/proc.go:305 +0xb0 fp=0xc000040fe0 sp=0xc000040fb0 pc=0x47e1d0 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000040fe8 sp=0xc000040fe0 pc=0x4ad1a1 created by runtime.init.6 /usr/local/go/src/runtime/proc.go:293 +0x25 goroutine 3 [GC sweep wait]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000041780 sp=0xc000041760 pc=0x47e396 runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:387 runtime.bgsweep(0x0?) /usr/local/go/src/runtime/mgcsweep.go:278 +0x8e fp=0xc0000417c8 sp=0xc000041780 pc=0x46a5ae runtime.gcenable.func1() /usr/local/go/src/runtime/mgc.go:178 +0x26 fp=0xc0000417e0 sp=0xc0000417c8 pc=0x45f866 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000417e8 sp=0xc0000417e0 pc=0x4ad1a1 created by runtime.gcenable /usr/local/go/src/runtime/mgc.go:178 +0x6b goroutine 4 [GC scavenge wait]: runtime.gopark(0xc000068000?, 0xd00e88?, 0x1?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000041f70 sp=0xc000041f50 pc=0x47e396 runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:387 runtime.(*scavengerState).park(0x112cc00) /usr/local/go/src/runtime/mgcscavenge.go:400 +0x53 fp=0xc000041fa0 sp=0xc000041f70 pc=0x4684d3 runtime.bgscavenge(0x0?) /usr/local/go/src/runtime/mgcscavenge.go:628 +0x45 fp=0xc000041fc8 sp=0xc000041fa0 pc=0x468aa5 runtime.gcenable.func2() /usr/local/go/src/runtime/mgc.go:179 +0x26 fp=0xc000041fe0 sp=0xc000041fc8 pc=0x45f806 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000041fe8 sp=0xc000041fe0 pc=0x4ad1a1 created by runtime.gcenable /usr/local/go/src/runtime/mgc.go:179 +0xaa goroutine 18 [finalizer wait]: runtime.gopark(0x1a0?, 0x112d8e0?, 0xe0?, 0x24?, 0xc000040770?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000040628 sp=0xc000040608 pc=0x47e396 runtime.runfinq() /usr/local/go/src/runtime/mfinal.go:193 +0x107 fp=0xc0000407e0 sp=0xc000040628 pc=0x45e8a7 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000407e8 sp=0xc0000407e0 pc=0x4ad1a1 created by runtime.createfing /usr/local/go/src/runtime/mfinal.go:163 +0x45 goroutine 19 [select]: runtime.gopark(0xc00003c720?, 0x2?, 0x0?, 0x0?, 0xc00003c67c?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000053cd0 sp=0xc000053cb0 pc=0x47e396 runtime.selectgo(0xc000053f20, 0xc00003c678, 0x0?, 0x0, 0x0?, 0x1) /usr/local/go/src/runtime/select.go:327 +0x7be fp=0xc000053e10 sp=0xc000053cd0 pc=0x48db7e github.com/go-skynet/LocalAI/api.(*galleryApplier).start.func1() /build/api/gallery.go:57 +0xf7 fp=0xc000053fe0 sp=0xc000053e10 pc=0x93d6b7 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000053fe8 sp=0xc000053fe0 pc=0x4ad1a1 created by github.com/go-skynet/LocalAI/api.(*galleryApplier).start /build/api/gallery.go:55 +0xaa goroutine 20 [sleep]: runtime.gopark(0x1c9876a0453?, 0xc00003cf88?, 0xa5?, 0xd8?, 0xc0001d2850?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc00003cf58 sp=0xc00003cf38 pc=0x47e396 time.Sleep(0x2540be400) /usr/local/go/src/runtime/time.go:195 +0x135 fp=0xc00003cf98 sp=0xc00003cf58 pc=0x4aa015 github.com/valyala/fasthttp.(*workerPool).Start.func2() /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:67 +0x56 fp=0xc00003cfe0 sp=0xc00003cf98 pc=0x856016 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00003cfe8 sp=0xc00003cfe0 pc=0x4ad1a1 created by github.com/valyala/fasthttp.(*workerPool).Start /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:59 +0xdd goroutine 5 [IO wait]: runtime.gopark(0x0?, 0xb?, 0x0?, 0x0?, 0x7?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000163a28 sp=0xc000163a08 pc=0x47e396 runtime.netpollblock(0x4c0c05?, 0x449c2f?, 0x0?) /usr/local/go/src/runtime/netpoll.go:527 +0xf7 fp=0xc000163a60 sp=0xc000163a28 pc=0x476cf7 internal/poll.runtime_pollWait(0x7f1fac468a68, 0x72) /usr/local/go/src/runtime/netpoll.go:306 +0x89 fp=0xc000163a80 sp=0xc000163a60 pc=0x4a7a49 internal/poll.(*pollDesc).wait(0xc00007a000?, 0xc000204000?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc000163aa8 sp=0xc000163a80 pc=0x51e7b2 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Read(0xc00007a000, {0xc000204000, 0x1000, 0x1000}) /usr/local/go/src/internal/poll/fd_unix.go:167 +0x299 fp=0xc000163b40 sp=0xc000163aa8 pc=0x51fb99 net.(*netFD).Read(0xc00007a000, {0xc000204000?, 0xc00007e088?, 0xc00007e000?}) /usr/local/go/src/net/fd_posix.go:55 +0x29 fp=0xc000163b88 sp=0xc000163b40 pc=0x5a76c9 net.(*conn).Read(0xc000014010, {0xc000204000?, 0xc000014010?, 0xc000205000?}) /usr/local/go/src/net/net.go:183 +0x45 fp=0xc000163bd0 sp=0xc000163b88 pc=0x5b6bc5 net.(*TCPConn).Read(0xc0001ae5e0?, {0xc000204000?, 0x83a3cf?, 0x83d525?}) :1 +0x29 fp=0xc000163c00 sp=0xc000163bd0 pc=0x5c9569 bufio.(*Reader).fill(0xc00006c120) /usr/local/go/src/bufio/bufio.go:106 +0xff fp=0xc000163c38 sp=0xc000163c00 pc=0x60b0bf bufio.(*Reader).Peek(0xc00006c120, 0x1) /usr/local/go/src/bufio/bufio.go:144 +0x5d fp=0xc000163c58 sp=0xc000163c38 pc=0x60b21d github.com/valyala/fasthttp.(*Server).serveConn(0xc0001ae400, {0xd0b240?, 0xc000014010}) /go/pkg/mod/github.com/valyala/[email protected]/server.go:2176 +0x58e fp=0xc000163ec8 sp=0xc000163c58 pc=0x84a40e github.com/valyala/fasthttp.(*Server).serveConn-fm({0xd0b240?, 0xc000014010?}) :1 +0x39 fp=0xc000163ef0 sp=0xc000163ec8 pc=0x85a919 github.com/valyala/fasthttp.(*workerPool).workerFunc(0xc0001d2820, 0xc000024060) /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:224 +0xa9 fp=0xc000163fa0 sp=0xc000163ef0 pc=0x856b49 github.com/valyala/fasthttp.(*workerPool).getCh.func1() /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:196 +0x38 fp=0xc000163fe0 sp=0xc000163fa0 pc=0x8568b8 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000163fe8 sp=0xc000163fe0 pc=0x4ad1a1 created by github.com/valyala/fasthttp.(*workerPool).getCh /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:195 +0x1b0 goroutine 6 [sleep]: runtime.gopark(0x1c9cc0259d4?, 0xb88a60?, 0xb0?, 0x8a?, 0xc0001d2820?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000054f88 sp=0xc000054f68 pc=0x47e396 time.Sleep(0x3b9aca00) /usr/local/go/src/runtime/time.go:195 +0x135 fp=0xc000054fc8 sp=0xc000054f88 pc=0x4aa015 github.com/valyala/fasthttp.updateServerDate.func1() /go/pkg/mod/github.com/valyala/[email protected]/header.go:2247 +0x1e fp=0xc000054fe0 sp=0xc000054fc8 pc=0x856f9e runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000054fe8 sp=0xc000054fe0 pc=0x4ad1a1 created by github.com/valyala/fasthttp.updateServerDate /go/pkg/mod/github.com/valyala/[email protected]/header.go:2245 +0x25 goroutine 7 [chan receive]: runtime.gopark(0x8?, 0xb7e560?, 0x1?, 0xf6?, 0x7f1fac468978?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000055e38 sp=0xc000055e18 pc=0x47e396 runtime.chanrecv(0xc00006c1e0, 0xc000055f80, 0x1) /usr/local/go/src/runtime/chan.go:583 +0x49d fp=0xc000055ec8 sp=0xc000055e38 pc=0x44d35d runtime.chanrecv2(0xc0001d2820?, 0xc000024100?) /usr/local/go/src/runtime/chan.go:447 +0x18 fp=0xc000055ef0 sp=0xc000055ec8 pc=0x44ce98 github.com/valyala/fasthttp.(*workerPool).workerFunc(0xc0001d2820, 0xc000024100) /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:219 +0x65 fp=0xc000055fa0 sp=0xc000055ef0 pc=0x856b05 github.com/valyala/fasthttp.(*workerPool).getCh.func1() /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:196 +0x38 fp=0xc000055fe0 sp=0xc000055fa0 pc=0x8568b8 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000055fe8 sp=0xc000055fe0 pc=0x4ad1a1 created by github.com/valyala/fasthttp.(*workerPool).getCh /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:195 +0x1b0 rax 0x7540b99f rbx 0x14c5400 rcx 0x0 rdx 0x2afdf rdi 0x1 rsi 0x753e09c0 rbp 0x14e5400 rsp 0x7f1facc8f5b0 r8 0x7ffeb659b080 r9 0x7af r10 0x7ffeb659b090 r11 0x5cb2a r12 0x1505400 r13 0x1485400 r14 0x0 r15 0x0 rip 0xad522d rflags 0x10202 cs 0x33 fs 0x0 gs 0x0
Just tried with "Bert embeddings" too, same error. It crashes ass soon as I send the post request.
I wonder what is your input, can you follow this example? It should work: https://github.com/go-skynet/LocalAI/blob/master/README.md#example-use-gpt4all-j-model
I have the same problem. I googled a bit and there are a few things that point to the cpu not having the avx2 instruction (https://github.com/flashlight/wav2letter/issues/604). I don't know enough about cpus or the interals of the code to be sure, but it makes sense to me. Probably a cflag in the build would fix it, or to buy a new computer (pre 2013 cpus seem to have the problem). Obviously I would like it much more to have the software-solution, but... well... EDIT: My cpu doesnt have AVX at all (Xeon X5650) - maybe thats the problem and not avx2, which is "only" faster avx, as far as i understand.
I wonder what is your input, can you follow this example? It should work: https://github.com/go-skynet/LocalAI/blob/master/README.md#example-use-gpt4all-j-model
In fact, I strictly followed this example as I mentioned at the first reply. Every time I use curl to post, it crashes.
same here, but i investigated a bit more and i am quite sure right now that its the instructionset that is the problem. compiling it all for ourselfs (no docker, i mean everything that is used for localAI) should fix it, but i didnt do that as of now - i have had better/more important things to do. my guess is that its a binary that is loaded from somewhere that requires this instruction. In my case its probably the vzeroupper instruction, as "seen" in my error, which is slightly different from that of @CaioLuppo (i get " instruction bytes: 0xc5 0xf1 0xef 0xc9 0xc5 0xfd 0x6f 0x5 0x18 0xab 0x24 0x0 0x49 0xc7 0x86 0x80"; the first one should be: 0xc5 =>11000101 => vzeroupper => AVX instruction set).
Yeah, I think AVX is really the problem here. I tried the "AVX only" version of the Gpt4All software and it worked, but I don't think there is such an option in LocalAI...
@Aisuko, is there any fix we can do to make it work with sorta like Gpt4All AVX only?
confirming that issue occurs when there is no avx2 support.
i have tried to run vicuna based model on different server configurations:
- cpu with avx2 support — OK
- cpu with avx only — FAILED, but llama.cpp runs well
I think there is a problem while go-llama linkings building process.
confirming that issue occurs when there is no avx2 support.
i have tried to run vicuna based model on different server configurations:
* cpu with avx2 support — OK * cpu with avx only — FAILED, but llama.cpp runs wellI think there is a problem while go-llama linkings building process.
Can you try building LocalAI from source with:
CMAKE_ARGS="-DLLAMA_AVX2=OFF" make build
That should build LocalAI without avx2 on llama.cpp
confirming that issue occurs when there is no avx2 support. i have tried to run vicuna based model on different server configurations:
* cpu with avx2 support — OK * cpu with avx only — FAILED, but llama.cpp runs wellI think there is a problem while go-llama linkings building process.
Can you try building LocalAI from source with:
CMAKE_ARGS="-DLLAMA_AVX2=OFF" make buildThat should build LocalAI without avx2 on llama.cpp
That didn't work. Still getting the same error. Does it mean that problem is not in AVX2 bindings?
possible, might be other CPU instruction flag missing. You can try disabling more instructions set like this (although, will be quite slow):
CMAKE_ARGS="-DLLAMA_F16C=OFF -DLLAMA_AVX512=OFF -DLLAMA_AVX2=OFF -DLLAMA_FMA=OFF" make build
Hey, I have something to add to this. I did run into this issue with an AVX only CPU. I wasn't able to get docker to work but I could build the LocalAI binary using CMAKE_ARGS="-DLLAMA_AVX2=OFF" and it didn't have this issue outside of docker. When I try to pass this flag to the docker build the build fails with this instead
#0 524.3 g++: warning: utils.o: linker input file unused because linking not done
#0 524.3 /go/pkg/mod/github.com/mattn/[email protected]/isatty_tcgets.go:7:8: missing go.sum entry for module providing package golang.org/x/sys/unix (imported by github.com/mattn/go-isatty); to add:
#0 524.3 go get github.com/mattn/[email protected]
Is this an issue with the go docker container?
I have the same issue. My CPU is based on Sandy Bridge microarchitecture which doesn't support F16C extension.
Based on the error I'm getting the instruction set is 'c4 e2 79 13 c9 vcvtph2ps xmm1,xmm1'.
CMAKE_ARGS=-DLLAMA_F16C=OFF has no effect, because there are plenty of dependencies which always build with "-mf16c" flag regardless whether CPU supports that extension or not.
I'm currently trying to figure out how to remove those flags from makefiles and rebuild.
find ./ -type f -exec sed -i 's/-mf16c//g' {} \;
This is fixed for me in the latest release as long as I do docker compose with build vs pull. I didn't have to put any flags and those didn't seem to help before anyway.
I think the issue is the change to llama.cpp, which introduced support for AVX only, with new cmake flag added to control this feature.
I'm running local-ai on X5550, and it was failing until I used the following options:
CMAKE_ARGS="-DLLAMA_F16C=OFF -DLLAMA_AVX512=OFF -DLLAMA_AVX2=OFF -DLLAMA_AVX=OFF -DLLAMA_FMA=OFF -DLLAMA_F16C=OFF"
Note -DLLAMA_AVX=OFF, this option is not mentioned anywhere on local-ai doco, or github. I also used -DLLAMA_F16C=OFF for good measure, as my CPU doesn't support it, and it looks like it always on by default.
This two changes fixed local-ai and now I can run it with no problem.
Namely issue is this PR - https://github.com/ggerganov/llama.cpp/pull/617, which added AVX support, along with new compile options to control it, with a default ON. Those new options are not part of REBUILD=true machinery, as such you still get llama compiled with AVX support, not AVX2, or AVX512, but plain AVX.
I also used
-DLLAMA_F16C=OFFfor...
Greetings! Please check the very beginning of the variable value ^^
After comparing results on two different computers, one of which has avx2 whereas the other doesn't, I also came to the conclusion that it had to have something to do with these CPU flags (I wish I had found this thread earlier but I was searching for the output I got on the command line "error reading from server: EOF" rather than the one I later found in the debug info which could have led me here sooner...).
So I tried to rebuild local-ai in the docker but it complained about not finding something (some libraries?). So instead of investing more time in this container, knowing all would be gone with its next recreation, I decided to build local-ai directly in my VM, without docker.
With avx2 support switched off, the build eventually worked and the error is gone now.
I think the issue is the change to llama.cpp, which introduced support for AVX only, with new cmake flag added to control this feature.
I'm running local-ai on X5550, and it was failing until I used the following options:
CMAKE_ARGS="-DLLAMA_F16C=OFF -DLLAMA_AVX512=OFF -DLLAMA_AVX2=OFF -DLLAMA_AVX=OFF -DLLAMA_FMA=OFF -DLLAMA_F16C=OFF"Note -DLLAMA_AVX=OFF, this option is not mentioned anywhere on local-ai doco, or github. I also used -DLLAMA_F16C=OFF for good measure, as my CPU doesn't support it, and it looks like it always on by default.
This two changes fixed local-ai and now I can run it with no problem.
Could you help to please show me which file i could put "`CMAKE_ARGS=...' into to fix the problem? THank you.
Just in case, GitHub supports spoilers for huge pages of wonderful content.
Thank you. I tried but still getting error
Trying this on yet another machine. This one has avx, avx2, f16c and fma.
But I am still getting this error when using this docker image quay.io/go-skynet/local-ai:v2.2.0-cublas-cuda12-ffmpeg
Do I also need AVX512 to run the image?
(It would be great, if someone could point me to where the requirements for the docker images are documented.)
Thanks!
This is fixed for me in the latest release as long as I do docker compose with build vs pull. I didn't have to put any flags and those didn't seem to help before anyway.
@stereotypy Could you share your compose yaml? It would be greatly appreciated.
This is fixed for me in the latest release as long as I do docker compose with build vs pull. I didn't have to put any flags and those didn't seem to help before anyway.
@stereotypy Could you share your compose yaml? It would be greatly appreciated.
this should be fixed - can you share the full logs?