jan
jan copied to clipboard
janhq
bug:Load model failed with error TypeError: fetch failed [AVX]
hello, I can't start any model.
{"timestamp":1712753736,"level":"INFO","function":"loadModelImpl","line":642,"message":"system info","n_threads":25,"total_threads":48,"system_info":"AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 0 | VSX = 0 | MATMUL_INT8 = 0 | "} [1712753736] [D:\a\nitro\nitro\controllers\llamaCPP.h: 547][llama_server_context::load_model] Multi Modal Mode Enabled 2024-04-10T12:55:36.773Z [NITRO]::Debug: Nitro exited with code: 3221225501 2024-04-10T12:55:38.281Z [NITRO]::Error: Load model failed with error TypeError: fetch failed 2024-04-10T12:55:38.283Z [NITRO]::Error: TypeError: fetch failed 2024-04-10T12:58:43.294Z [NITRO]::Debug: Request to kill Nitro 2024-04-10T12:58:43.333Z [NITRO]::Debug: Nitro process is terminated 2024-04-15T10:29:32.659Z [SPECS]::Version: 0.4.11 2024-04-15T10:29:32.661Z [SPECS]::CPUs: [{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":208328,"nice":0,"sys":655359,"idle":481147187,"irq":21234}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":142921,"nice":0,"sys":224078,"idle":481643875,"irq":156}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":2091468,"nice":0,"sys":524640,"idle":479394765,"irq":132250}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":35281,"nice":0,"sys":30390,"idle":481944828,"irq":3687}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":135609,"nice":0,"sys":639625,"idle":481235265,"irq":8953}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":44625,"nice":0,"sys":56937,"idle":481908937,"irq":109}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":95843,"nice":0,"sys":404671,"idle":481509984,"irq":4359}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":22140,"nice":0,"sys":23906,"idle":481964437,"irq":140}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":120078,"nice":0,"sys":282578,"idle":481607828,"irq":2281}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":22578,"nice":0,"sys":57375,"idle":481930531,"irq":93}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":1466546,"nice":0,"sys":4599828,"idle":475944109,"irq":13203}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":23593,"nice":0,"sys":68734,"idle":481918156,"irq":140}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":472687,"nice":0,"sys":987984,"idle":480549812,"irq":24093}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":25171,"nice":0,"sys":17781,"idle":481967515,"irq":31}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":211328,"nice":0,"sys":388109,"idle":481411031,"irq":1640}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":31812,"nice":0,"sys":17734,"idle":481960921,"irq":125}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":135187,"nice":0,"sys":218234,"idle":481657046,"irq":609}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":46953,"nice":0,"sys":95031,"idle":481868484,"irq":250}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":92343,"nice":0,"sys":218703,"idle":481699406,"irq":515}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":37546,"nice":0,"sys":23062,"idle":481949828,"irq":109}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":121015,"nice":0,"sys":189406,"idle":481700031,"irq":390}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":31546,"nice":0,"sys":30781,"idle":481948109,"irq":140}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":201562,"nice":0,"sys":225921,"idle":481582968,"irq":6687}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":18671,"nice":0,"sys":18750,"idle":481973000,"irq":156}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":323171,"nice":0,"sys":592781,"idle":481094484,"irq":48609}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":80765,"nice":0,"sys":57906,"idle":481871750,"irq":1281}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":2965187,"nice":0,"sys":767062,"idle":478278171,"irq":12343}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":34937,"nice":0,"sys":24640,"idle":481950828,"irq":140}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":216093,"nice":0,"sys":385937,"idle":481408390,"irq":968}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":84421,"nice":0,"sys":152484,"idle":481773484,"irq":3187}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":315609,"nice":0,"sys":380671,"idle":481314109,"irq":5765}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":42359,"nice":0,"sys":20281,"idle":481947750,"irq":218}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":274656,"nice":0,"sys":263515,"idle":481472203,"irq":2125}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":30234,"nice":0,"sys":10390,"idle":481969750,"irq":93}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":115687,"nice":0,"sys":220328,"idle":481674359,"irq":1015}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":39390,"nice":0,"sys":18640,"idle":481952312,"irq":156}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":271281,"nice":0,"sys":328718,"idle":481410359,"irq":1531}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":38609,"nice":0,"sys":16640,"idle":481955093,"irq":171}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":242328,"nice":0,"sys":227765,"idle":481540250,"irq":1125}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":37093,"nice":0,"sys":48031,"idle":481925218,"irq":6093}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":168078,"nice":0,"sys":140296,"idle":481701968,"irq":1390}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":33703,"nice":0,"sys":12765,"idle":481963859,"irq":234}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":133250,"nice":0,"sys":95484,"idle":481781593,"irq":1062}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":68000,"nice":0,"sys":19859,"idle":481922468,"irq":265}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":200687,"nice":0,"sys":110484,"idle":481699140,"irq":875}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":96296,"nice":0,"sys":22375,"idle":481891640,"irq":312}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":186000,"nice":0,"sys":134703,"idle":481689593,"irq":4468}},{"model":"Intel(R) Xeon(R) CPU E5-2696 v2 @ 2.50GHz","speed":2500,"times":{"user":141531,"nice":0,"sys":72468,"idle":481796281,"irq":2687}}] 2024-04-15T10:29:32.662Z [SPECS]::Endianness: LE 2024-04-15T10:29:32.662Z [SPECS]::Machine: x86_64 2024-04-15T10:29:32.663Z [SPECS]::Parallelism: 48 2024-04-15T10:29:32.663Z [SPECS]::Free Mem: 96012664832 2024-04-15T10:29:32.663Z [SPECS]::Total Mem: 103012655104 2024-04-15T10:29:32.664Z [SPECS]::OS Version: Windows 10 Pro 2024-04-15T10:29:32.664Z [SPECS]::OS Platform: win32 2024-04-15T10:29:32.665Z [SPECS]::OS Release: 10.0.22621 2024-04-15T10:29:32.666Z [SPECS]::0, 23040, Tesla P40
2024-04-15T10:29:32.672Z [APP]::{"notify":true,"run_mode":"cpu","nvidia_driver":{"exist":true,"version":"538.33"},"cuda":{"exist":true,"version":"12"},"gpus":[{"id":"0","vram":"23040","name":"Tesla P40","arch":"unknown"}],"gpu_highest_vram":"0","gpus_in_use":["0"],"is_initial":false,"vulkan":false} 2024-04-15T10:29:50.444Z [NITRO]::CPU informations - 25 2024-04-15T10:29:50.447Z [NITRO]::Debug: Request to kill Nitro 2024-04-15T10:29:50.559Z [NITRO]::Debug: Nitro process is terminated 2024-04-15T10:29:50.563Z [NITRO]::Debug: Spawning Nitro subprocess... 2024-04-15T10:29:50.564Z [NITRO]::Debug: Spawn nitro at path: C:\Users\jason\jan\extensions@janhq\inference-nitro-extension\dist\bin\win-cpu\nitro.exe, and args: 1,127.0.0.1,3928 2024-04-15T10:29:50.816Z [NITRO]::Debug: [93m [94m [93m [94m [93m [94m [93m_[94m_[93m_[94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m_[94m_[93m_[94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m_[94m_[93m_[94m [93m [94m [93m [94m [0m [93m [94m [93m [94m [93m [94m/[93m_[94m_[93m/[94m [93m [94m [93m [94m [93m [94m [93m [94m_[93m_[94m_[93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m_[93m_[94m_[93m [94m [93m [94m [93m [94m [93m [94m [93m/[94m [93m [94m/[93m\[94m [93m [94m [93m [94m [93m [94m [93m [94m [93m/[94m [93m [94m/[93m\[94m [93m [94m [93m [0m [93m [94m [93m [94m [93m [94m\[93m [94m [93m\[94m:[93m\[94m [93m [94m [93m [94m [93m [94m/[93m [94m [93m/[94m\[93m [94m [93m [94m [93m [94m [93m [94m [93m [94m/[93m [94m [93m/[94m\[93m [94m [93m [94m [93m [94m [93m/[94m [93m [94m/[93m:[94m:[93m\[94m [93m [94m [93m [94m [93m [94m [93m/[94m [93m [94m/[93m:[94m:[93m\[94m [93m [94m [0m [93m [94m [93m [94m [93m [94m [93m\[94m [93m [94m\[93m:[94m\[93m [94m [93m [94m [93m/[94m [93m [94m/[93m:[94m/[93m [94m [93m [94m [93m [94m [93m [94m [93m/[94m [93m [94m/[93m:[94m/[93m [94m [93m [94m [93m [94m/[93m [94m [93m/[94m:[93m/[94m\[93m:[94m\[93m [94m [93m [94m [93m [94m/[93m [94m [93m/[94m:[93m/[94m\[93m:[94m\[93m [94m [0m [93m [94m [93m_[94m_[93m_[94m_[93m_[94m\[93m_[94m_[93m\[94m:[93m\[94m [93m [94m/[93m_[94m_[93m/[94m:[93m:[94m\[93m [94m [93m [94m [93m [94m [93m [94m/[93m [94m [93m/[94m:[93m/[94m [93m [94m [93m [94m [93m/[94m [93m [94m/[93m:[94m/[93m [94m [93m\[94m:[93m\[94m [93m [94m [93m/[94m [93m [94m/[93m:[94m/[93m [94m [93m\[94m:[93m\[94m [0m [93m [94m/[93m_[94m_[93m/[94m:[93m:[94m:[93m:[94m:[93m:[94m:[93m:[94m\[93m [94m\[93m_[94m_[93m\[94m/[93m\[94m:[93m\[94m_[93m_[94m [93m [94m [93m/[94m [93m [94m/[93m:[94m:[93m\[94m [93m [94m [93m [94m/[93m_[94m_[93m/[94m:[93m/[94m [93m/[94m:[93m/[94m_[93m_[94m_[93m [94m/[93m_[94m_[93m/[94m:[93m/[94m [93m\[94m_[93m_[94m\[93m:[94m\[0m [93m [94m\[93m [94m [93m\[94m:[93m\[94m~[93m~[94m\[93m~[94m~[93m\[94m/[93m [94m [93m [94m [93m\[94m [93m [94m\[93m:[94m\[93m/[94m\[93m [94m/[93m_[94m_[93m/[94m:[93m/[94m\[93m:[94m\[93m [94m [93m [94m\[93m [94m [93m\[94m:[93m\[94m/[93m:[94m:[93m:[94m:[93m:[94m/[93m [94m\[93m [94m [93m\[94m:[93m\[94m [93m/[94m [93m [94m/[93m:[94m/[0m [93m [94m [93m\[94m [93m [94m\[93m:[94m\[93m [94m [93m~[94m~[93m~[94m [93m [94m [93m [94m [93m [94m\[93m_[94m_[93m\[94m:[93m:[94m/[93m [94m\[93m_[94m_[93m\[94m/[93m [94m [93m\[94m:[93m\[94m [93m [94m [93m\[94m [93m [94m\[93m:[94m:[93m/[94m~[93m~[94m~[93m~[94m [93m [94m [93m\[94m [93m [94m\[93m:[94m\[93m [94m [93m/[94m:[93m/[94m [0m [93m [94m [93m [94m\[93m [94m [93m\[94m:[93m\[94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m/[93m_[94m_[93m/[94m:[93m/[94m [93m [94m [93m [94m [93m [94m [93m\[94m [93m [94m\[93m:[94m\[93m [94m [93m [94m\[93m [94m [93m\[94m:[93m\[94m [93m [94m [93m [94m [93m [94m [93m [94m\[93m [94m [93m\[94m:[93m\[94m/[93m:[94m/[93m [94m [0m [93m [94m [93m [94m [93m\[94m [93m [94m\[93m:[94m\[93m [94m [93m [94m [93m [94m [93m [94m [93m [94m\[93m_[94m_[93m\[94m/[93m [94m [93m [94m [93m [94m [93m [94m [93m [94m\[93m_[94m_[93m\[94m/[93m [94m [93m [94m [93m\[94m [93m [94m\[93m:[94m\[93m [94m [93m [94m [93m [94m [93m [94m [93m\[94m [93m [94m\[93m:[94m:[93m/[94m [93m [94m [0m [93m [94m [93m [94m [93m [94m\[93m_[94m_[93m\[94m/[93m [94m [93m [94m [93m [ 2024-04-15T10:29:51.183Z [NITRO]::Debug: Nitro is ready 2024-04-15T10:29:51.184Z [NITRO]::Debug: Loading model with params {"vision_model":true,"text_model":false,"ctx_len":4096,"prompt_template":"\n### Instruction:\n{prompt}\n### Response:\n","llama_model_path":"C:\Users\jason\jan\models\llava-13b\llava-v1.6-vicuna-13b.Q4_K_M.gguf","mmproj":"C:\Users\jason\jan\models\llava-13b\mmproj-model-f16.gguf","user_prompt":"\n### Instruction:\n","ai_prompt":"\n### Response:\n","cpu_threads":25,"ngl":100} 2024-04-15T10:29:51.222Z [NITRO]::Debug: 94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m [93m [94m\[93m_[94m_[93m\[94m/[93m [94m [93m [94m [93m [94m [93m [94m [93m [94m\[93m_[94m_[93m\[94m/[93m [94m [93m [94m [0m [0m20240415 10:29:50.816000 UTC 17576 INFO Nitro version: - main.cc:50 20240415 10:29:50.818000 UTC 17576 INFO Server started, listening at: 127.0.0.1:3928 - main.cc:54 20240415 10:29:50.818000 UTC 17576 INFO Please load your model - main.cc:55 20240415 10:29:50.818000 UTC 17576 INFO Number of thread is:48 - main.cc:62 20240415 10:29:51.205000 UTC 19124 INFO MMPROJ FILE detected, multi-model enabled! - llamaCPP.cc:579 {"timestamp":1713176991,"level":"INFO","function":"LoadModelImpl","line":650,"message":"system info","n_threads":25,"total_threads":48,"system_info":"AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 0 | VSX = 0 | MATMUL_INT8 = 0 | "}
2024-04-15T10:29:51.248Z [NITRO]::Debug: Nitro exited with code: 3221225501 2024-04-15T10:29:52.156Z [NITRO]::Error: Load model failed with error TypeError: fetch failed 2024-04-15T10:29:52.156Z [NITRO]::Error: TypeError: fetch failed
