private-gpt
private-gpt copied to clipboard
zylon-ai
Illegal instruction (core dumped)
root@fd13db08a5d7:/usr/src/app# python ./privateGPT.py
llama.cpp: loading model from models/ggml-model-q4_0.bin
llama.cpp: can't use mmap because tensors are not aligned; convert to new format to avoid this
llama_model_load_internal: format = 'ggml' (old version with low tokenizer quality and no mmap support)
llama_model_load_internal: n_vocab = 32000
llama_model_load_internal: n_ctx = 1000
llama_model_load_internal: n_embd = 4096
llama_model_load_internal: n_mult = 256
llama_model_load_internal: n_head = 32
llama_model_load_internal: n_layer = 32
llama_model_load_internal: n_rot = 128
llama_model_load_internal: ftype = 2 (mostly Q4_0)
llama_model_load_internal: n_ff = 11008
llama_model_load_internal: n_parts = 1
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size = 4113748.20 KB
llama_model_load_internal: mem required = 5809.33 MB (+ 2052.00 MB per state)
...................................................................................................
.
llama_init_from_file: kv self size = 1000.00 MB
AVX = 1 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |
Using embedded DuckDB with persistence: data will be stored in: db
Illegal instruction (core dumped)
I just follow the instructions in README, all the steps when well, pip install python ingest.py ...
when I try the last step python ./privateGPT.py I get this.
I run the code on an old xeon CPU (2012), will it be something missing on the CPU? I run 8 cores + 16GB ram
I run in docker with image python:3
Yeah, same here. Followed the instructions with the sample.
EDIT: After waiting for a bit, it worked but was very slow and did get some kind of an error. Here are the results.
llama.cpp: loading model from D:\PrivateGPT\models\ggml-model-q4_0.bin
llama.cpp: can't use mmap because tensors are not aligned; convert to new format to avoid this
llama_model_load_internal: format = 'ggml' (old version with low tokenizer quality and no mmap support)
llama_model_load_internal: n_vocab = 32000
llama_model_load_internal: n_ctx = 1000
llama_model_load_internal: n_embd = 4096
llama_model_load_internal: n_mult = 256
llama_model_load_internal: n_head = 32
llama_model_load_internal: n_layer = 32
llama_model_load_internal: n_rot = 128
llama_model_load_internal: ftype = 2 (mostly Q4_0)
llama_model_load_internal: n_ff = 11008
llama_model_load_internal: n_parts = 1
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size = 4113748.20 KB
llama_model_load_internal: mem required = 5809.33 MB (+ 2052.00 MB per state)
...................................................................................................
.
llama_init_from_file: kv self size = 1000.00 MB
AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |
Using embedded DuckDB with persistence: data will be stored in: db
gptj_model_load: loading model from 'D:\PrivateGPT\models\ggml-gpt4all-j-v1.3-groovy.bin' - please wait ...
gptj_model_load: n_vocab = 50400
gptj_model_load: n_ctx = 2048
gptj_model_load: n_embd = 4096
gptj_model_load: n_head = 16
gptj_model_load: n_layer = 28
gptj_model_load: n_rot = 64
gptj_model_load: f16 = 2
gptj_model_load: ggml ctx size = 4505.45 MB
gptj_model_load: memory_size = 896.00 MB, n_mem = 57344
gptj_model_load: ................................... done
gptj_model_load: model size = 3609.38 MB / num tensors = 285
Enter a query: What did the president say about russia?
llama_print_timings: load time = 504.66 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 646.91 ms / 10 tokens ( 64.69 ms per token)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: total time = 650.76 ms
gpt_tokenize: unknown token 'Γ'
gpt_tokenize: unknown token 'Ç'
gpt_tokenize: unknown token 'Ö'
gpt_tokenize: unknown token 'Γ'
gpt_tokenize: unknown token 'Ç'
gpt_tokenize: unknown token 'Ö'
gpt_tokenize: unknown token 'Γ'
gpt_tokenize: unknown token 'Ç'
gpt_tokenize: unknown token 'Ö'
gpt_tokenize: unknown token 'Γ'
gpt_tokenize: unknown token 'Ç'
gpt_tokenize: unknown token '£'
gpt_tokenize: unknown token 'Γ'
gpt_tokenize: unknown token 'Ç'
gpt_tokenize: unknown token '¥'
gpt_tokenize: unknown token 'Γ'
gpt_tokenize: unknown token 'Ç'
gpt_tokenize: unknown token 'Ö'
gpt_tokenize: unknown token 'Γ'
gpt_tokenize: unknown token 'Ç'
gpt_tokenize: unknown token 'Ö'
gpt_tokenize: unknown token 'Γ'
gpt_tokenize: unknown token 'Ç'
gpt_tokenize: unknown token 'Ö'
gpt_tokenize: unknown token 'Γ'
gpt_tokenize: unknown token 'Ç'
We are choking off Russia’s access to technology that will sap its economic strength and weaken its military for years to come.
Tonight I say to the Russian oligarchs and corrupt leaders who have bilked billions of dollars off this violent regime no more.
> source_documents\state_of_the_union.txt:
While it shouldn’t have taken something so terrible for people around the world to see what’s at stake now everyone sees it clearly.
We see the unity among leaders of nations and a more unified Europe a more unified West. And we see unity among the people who are gathering in cities in large crowds around the world even in Russia to demonstrate their support for Ukraine.
> source_documents\state_of_the_union.txt:
Please rise if you are able and show that, Yes, we the United States of America stand with the Ukrainian people.
Throughout our history we’ve learned this lesson when dictators do not pay a price for their aggression they cause more chaos.
They keep moving.
And the costs and the threats to America and the world keep rising.
That’s why the NATO Alliance was created to secure peace and stability in Europe after World War 2.
> source_documents\state_of_the_union.txt:
We prepared extensively and carefully.
We spent months building a coalition of other freedom-loving nations from Europe and the Americas to Asia and Africa to confront Putin.
I spent countless hours unifying our European allies. We shared with the world in advance what we knew Putin was planning and precisely how he would try to falsely justify his aggression.
We countered Russia’s lies with truth.
And now that he has acted the free world is holding him accountable.
Enter a query:```
I am getting the same error. I then tried to update to the latest ggml format but I still get the error.
python privateGPT.py
llama.cpp: loading model from models/ggml-model-q4_0.bin
llama_model_load_internal: format = ggjt v2 (latest)
llama_model_load_internal: n_vocab = 32000
llama_model_load_internal: n_ctx = 1000
llama_model_load_internal: n_embd = 4096
llama_model_load_internal: n_mult = 256
llama_model_load_internal: n_head = 32
llama_model_load_internal: n_layer = 32
llama_model_load_internal: n_rot = 128
llama_model_load_internal: ftype = 2 (mostly Q4_0)
llama_model_load_internal: n_ff = 11008
llama_model_load_internal: n_parts = 1
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size = 72.75 KB
llama_model_load_internal: mem required = 5809.34 MB (+ 2052.00 MB per state)
llama_init_from_file: kv self size = 1000.00 MB
AVX = 1 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |
Using embedded DuckDB with persistence: data will be stored in: db
Illegal instruction (core dumped)
Then Looking a dmesg I saw the following
[ 1006.633763] traps: python[7629] trap invalid opcode ip:7f50f5f1f776 sp:7fff6f8ac370 error:0 in _pyllamacpp.cpython-310-x86_64-linux-gnu.so[7f50f5edb000+64000] [ 1914.619589] traps: python[7773] trap invalid opcode ip:7f7869b1f776 sp:7ffc6d97bfe0 error:0 in _pyllamacpp.cpython-310-x86_64-linux-gnu.so[7f7869adb000+64000] [ 2127.314608] traps: python[7826] trap invalid opcode ip:7f1fdb287776 sp:7ffd8f856810 error:0 in _pyllamacpp.cpython-310-x86_64-linux-gnu.so[7f1fdb243000+64000] [ 2524.098079] traps: python[8023] trap invalid opcode ip:7f3335916776 sp:7ffdc2830a00 error:0 in _pyllamacpp.cpython-310-x86_64-linux-gnu.so[7f33358d2000+64000] [ 2742.237852] traps: python[8070] trap invalid opcode ip:7fa4cdf08776 sp:7ffd1c4eb990 error:0 in _pyllamacpp.cpython-310-x86_64-linux-gnu.so[7fa4cdec4000+64000]
The iasue is not about GGML, since the most recent release on pip of llama-cpp-python is not the most upstream one.
If you can run it wirh valgrind (which will significantly reduce the speed) we can help better.
I did try running the valgrind, this is the latest code.. any pointer will help, trying to run on a ubuntu vm with python3.10
valgrind python3.10 privateGPT.py
==312712== Memcheck, a memory error detector
==312712== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==312712== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==312712== Command: python3.10 privateGPT.py
==312712==
==312712== Invalid read of size 8
==312712== at 0x40286C0: strncmp (strcmp.S:170)
==312712== by 0x400668D: is_dst (dl-load.c:216)
==312712== by 0x400810E: _dl_dst_count (dl-load.c:253)
==312712== by 0x400810E: expand_dynamic_string_token (dl-load.c:395)
==312712== by 0x40082B7: fillin_rpath.isra.0 (dl-load.c:483)
==312712== by 0x4008602: decompose_rpath (dl-load.c:654)
==312712== by 0x400A8C8: cache_rpath (dl-load.c:696)
==312712== by 0x400A8C8: cache_rpath (dl-load.c:677)
==312712== by 0x400A8C8: _dl_map_object (dl-load.c:2117)
==312712== by 0x4003494: openaux (dl-deps.c:64)
==312712== by 0x4B11C27: _dl_catch_exception (dl-error-skeleton.c:208)
==312712== by 0x4003C7B: _dl_map_object_deps (dl-deps.c:248)
==312712== by 0x400EA0E: dl_open_worker_begin (dl-open.c:592)
==312712== by 0x4B11C27: _dl_catch_exception (dl-error-skeleton.c:208)
==312712== by 0x400DF99: dl_open_worker (dl-open.c:782)
==312712== Address 0x4c768cd is 365 bytes inside a block of size 372 alloc'd
==312712== at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==312712== by 0x40271FF: malloc (rtld-malloc.h:56)
==312712== by 0x40271FF: strdup (strdup.c:42)
==312712== by 0x4008594: decompose_rpath (dl-load.c:629)
==312712== by 0x400A8C8: cache_rpath (dl-load.c:696)
==312712== by 0x400A8C8: cache_rpath (dl-load.c:677)
==312712== by 0x400A8C8: _dl_map_object (dl-load.c:2117)
==312712== by 0x4003494: openaux (dl-deps.c:64)
==312712== by 0x4B11C27: _dl_catch_exception (dl-error-skeleton.c:208)
==312712== by 0x4003C7B: _dl_map_object_deps (dl-deps.c:248)
==312712== by 0x400EA0E: dl_open_worker_begin (dl-open.c:592)
==312712== by 0x4B11C27: _dl_catch_exception (dl-error-skeleton.c:208)
==312712== by 0x400DF99: dl_open_worker (dl-open.c:782)
==312712== by 0x4B11C27: _dl_catch_exception (dl-error-skeleton.c:208)
==312712== by 0x400E34D: _dl_open (dl-open.c:883)
==312712==
==312712== Invalid read of size 8
==312712== at 0x40286C8: strncmp (strcmp.S:172)
==312712== by 0x400668D: is_dst (dl-load.c:216)
==312712== by 0x400810E: _dl_dst_count (dl-load.c:253)
==312712== by 0x400810E: expand_dynamic_string_token (dl-load.c:395)
==312712== by 0x40082B7: fillin_rpath.isra.0 (dl-load.c:483)
==312712== by 0x4008602: decompose_rpath (dl-load.c:654)
==312712== by 0x400A8C8: cache_rpath (dl-load.c:696)
==312712== by 0x400A8C8: cache_rpath (dl-load.c:677)
==312712== by 0x400A8C8: _dl_map_object (dl-load.c:2117)
==312712== by 0x4003494: openaux (dl-deps.c:64)
==312712== by 0x4B11C27: _dl_catch_exception (dl-error-skeleton.c:208)
==312712== by 0x4003C7B: _dl_map_object_deps (dl-deps.c:248)
==312712== by 0x400EA0E: dl_open_worker_begin (dl-open.c:592)
==312712== by 0x4B11C27: _dl_catch_exception (dl-error-skeleton.c:208)
==312712== by 0x400DF99: dl_open_worker (dl-open.c:782)
==312712== Address 0x4c768d5 is 1 bytes after a block of size 372 alloc'd
==312712== at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==312712== by 0x40271FF: malloc (rtld-malloc.h:56)
==312712== by 0x40271FF: strdup (strdup.c:42)
==312712== by 0x4008594: decompose_rpath (dl-load.c:629)
==312712== by 0x400A8C8: cache_rpath (dl-load.c:696)
==312712== by 0x400A8C8: cache_rpath (dl-load.c:677)
==312712== by 0x400A8C8: _dl_map_object (dl-load.c:2117)
==312712== by 0x4003494: openaux (dl-deps.c:64)
==312712== by 0x4B11C27: _dl_catch_exception (dl-error-skeleton.c:208)
==312712== by 0x4003C7B: _dl_map_object_deps (dl-deps.c:248)
==312712== by 0x400EA0E: dl_open_worker_begin (dl-open.c:592)
==312712== by 0x4B11C27: _dl_catch_exception (dl-error-skeleton.c:208)
==312712== by 0x400DF99: dl_open_worker (dl-open.c:782)
==312712== by 0x4B11C27: _dl_catch_exception (dl-error-skeleton.c:208)
==312712== by 0x400E34D: _dl_open (dl-open.c:883)
==312712==
==312712== Invalid read of size 8
==312712== at 0x40286C0: strncmp (strcmp.S:170)
==312712== by 0x400668D: is_dst (dl-load.c:216)
==312712== by 0x4007F79: _dl_dst_substitute (dl-load.c:295)
==312712== by 0x40082B7: fillin_rpath.isra.0 (dl-load.c:483)
==312712== by 0x4008602: decompose_rpath (dl-load.c:654)
==312712== by 0x400A8C8: cache_rpath (dl-load.c:696)
==312712== by 0x400A8C8: cache_rpath (dl-load.c:677)
==312712== by 0x400A8C8: _dl_map_object (dl-load.c:2117)
==312712== by 0x4003494: openaux (dl-deps.c:64)
==312712== by 0x4B11C27: _dl_catch_exception (dl-error-skeleton.c:208)
==312712== by 0x4003C7B: _dl_map_object_deps (dl-deps.c:248)
==312712== by 0x400EA0E: dl_open_worker_begin (dl-open.c:592)
==312712== by 0x4B11C27: _dl_catch_exception (dl-error-skeleton.c:208)
==312712== by 0x400DF99: dl_open_worker (dl-open.c:782)
==312712== Address 0x4c768cd is 365 bytes inside a block of size 372 alloc'd
==312712== at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==312712== by 0x40271FF: malloc (rtld-malloc.h:56)
==312712== by 0x40271FF: strdup (strdup.c:42)
==312712== by 0x4008594: decompose_rpath (dl-load.c:629)
==312712== by 0x400A8C8: cache_rpath (dl-load.c:696)
==312712== by 0x400A8C8: cache_rpath (dl-load.c:677)
==312712== by 0x400A8C8: _dl_map_object (dl-load.c:2117)
==312712== by 0x4003494: openaux (dl-deps.c:64)
==312712== by 0x4B11C27: _dl_catch_exception (dl-error-skeleton.c:208)
==312712== by 0x4003C7B: _dl_map_object_deps (dl-deps.c:248)
==312712== by 0x400EA0E: dl_open_worker_begin (dl-open.c:592)
==312712== by 0x4B11C27: _dl_catch_exception (dl-error-skeleton.c:208)
==312712== by 0x400DF99: dl_open_worker (dl-open.c:782)
==312712== by 0x4B11C27: _dl_catch_exception (dl-error-skeleton.c:208)
==312712== by 0x400E34D: _dl_open (dl-open.c:883)
==312712==
==312712== Invalid read of size 8
==312712== at 0x40286C8: strncmp (strcmp.S:172)
==312712== by 0x400668D: is_dst (dl-load.c:216)
==312712== by 0x4007F79: _dl_dst_substitute (dl-load.c:295)
==312712== by 0x40082B7: fillin_rpath.isra.0 (dl-load.c:483)
==312712== by 0x4008602: decompose_rpath (dl-load.c:654)
==312712== by 0x400A8C8: cache_rpath (dl-load.c:696)
==312712== by 0x400A8C8: cache_rpath (dl-load.c:677)
==312712== by 0x400A8C8: _dl_map_object (dl-load.c:2117)
==312712== by 0x4003494: openaux (dl-deps.c:64)
==312712== by 0x4B11C27: _dl_catch_exception (dl-error-skeleton.c:208)
==312712== by 0x4003C7B: _dl_map_object_deps (dl-deps.c:248)
==312712== by 0x400EA0E: dl_open_worker_begin (dl-open.c:592)
==312712== by 0x4B11C27: _dl_catch_exception (dl-error-skeleton.c:208)
==312712== by 0x400DF99: dl_open_worker (dl-open.c:782)
==312712== Address 0x4c768d5 is 1 bytes after a block of size 372 alloc'd
==312712== at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==312712== by 0x40271FF: malloc (rtld-malloc.h:56)
==312712== by 0x40271FF: strdup (strdup.c:42)
==312712== by 0x4008594: decompose_rpath (dl-load.c:629)
==312712== by 0x400A8C8: cache_rpath (dl-load.c:696)
==312712== by 0x400A8C8: cache_rpath (dl-load.c:677)
==312712== by 0x400A8C8: _dl_map_object (dl-load.c:2117)
==312712== by 0x4003494: openaux (dl-deps.c:64)
==312712== by 0x4B11C27: _dl_catch_exception (dl-error-skeleton.c:208)
==312712== by 0x4003C7B: _dl_map_object_deps (dl-deps.c:248)
==312712== by 0x400EA0E: dl_open_worker_begin (dl-open.c:592)
==312712== by 0x4B11C27: _dl_catch_exception (dl-error-skeleton.c:208)
==312712== by 0x400DF99: dl_open_worker (dl-open.c:782)
==312712== by 0x4B11C27: _dl_catch_exception (dl-error-skeleton.c:208)
==312712== by 0x400E34D: _dl_open (dl-open.c:883)
==312712==
==312712== Warning: set address range perms: large range [0x2a7fa000, 0x3b8d5000) (noaccess)
==312712== Warning: set address range perms: large range [0x2a800000, 0x3b6db000) (defined)
==312712== Warning: set address range perms: large range [0x59c87000, 0x6de28000) (noaccess)
==312712== Warning: set address range perms: large range [0x59e00000, 0x6dda1000) (defined)
==312712== Warning: set address range perms: large range [0x6dda1000, 0x86c54000) (defined)
==312712== Warning: set address range perms: large range [0x6ede2000, 0x81cd6000) (defined)
==312712== Warning: set address range perms: large range [0x86c54000, 0xacd14000) (defined)
==312712== Warning: set address range perms: large range [0x89bf6000, 0xac003000) (defined)
Traceback (most recent call last):
File "/home/user/workspace/ai/privateGPT/privateGPT.py", line 57, in
Oh well it feels like it's about memory, not sure about that though.
Maybe you should try to compile llama-cpp-python and langchain on your local machine. Check my repos for the Jupyter Notebook that contains the code that can guide you...
Indeed Gentlemen, after having unsuccessfully tried to compile this project in Windows, I have switched to WSL and am now getting a "illegal Instruction" error - an HW error for sure, however its quite a new processor. Intel Celeron 1.5GHz CPU J3455
I also get `Illegal instruction" on several machines. I don't think it's a hardware resource issue, as it fails the same on a 24-core Intel Xeon E5-2620 with 32GB RAM.
I'm using Python 3.11.3 in pyenv.
python -vvv privateGPT.py reveals it tripping up on ~/.pyenv/versions/privateGPT/lib/python3.11/site-packages/_pyllamacpp.cpython-311-x86_64-linux-gnu.so
I ran into this on an older machine. But when I tried it on a newer CPU, it worked successfully. I think the difference is if the CPU supports the AVX2 instruction. To quickly check for this:
$ cat /proc/cpuinfo | grep avx2
Your CPU needs to support avx2 instructions, otherwise executing privateGPT script will give you the error: "Illegal instruction (core dumped)". Tested on two of my computers where one has an AMD CPU with avx instruction set (Illegal instruction (core dumped)), and on another INTEL CPU with avx and avx2 instruction set. (it worked). Solutions: https://tech.amikelive.com/node-887/how-to-resolve-error-illegal-instruction-core-dumped-when-running-import-tensorflow-in-a-python-program/
Brilliant, many thanks, however this project does not rely on TensorFlow -> would be great knowing what other libraries could possible be causing this error and which should we downgrade.
On Fri, May 19, 2023 at 10:52 AM ubuntinux @.***> wrote:
Your CPU needs to support avx2 instructions, otherwise executing privateGPT script will give you the error: "Illegal instruction (core dumped)". Tested on two of my computers where one has an AMD CPU with avx instruction set (Illegal instruction (core dumped)), and on another INTEL CPU with avx and avx2 instruction set. (it worked). Solutions: https://tech.amikelive.com/node-887/how-to-resolve-error-illegal-instruction-core-dumped-when-running-import-tensorflow-in-a-python-program/
— Reply to this email directly, view it on GitHub https://github.com/imartinez/privateGPT/issues/203#issuecomment-1554321796, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABMBTKEGLVOUJDLJOGCJOXLXG47GXANCNFSM6AAAAAAYDMJHC4 . You are receiving this because you commented.Message ID: @.***>
Following this guide https://github.com/nomic-ai/pygpt4all/issues/71 If it still fails git clone --recursive https://github.com/abdeladim-s/pygptj.git and install this with pip install e .
I've tried with all compiler options set to OFF in the CMakeLists.txt for pyllamacpp, and am still seeing the error. 🙁
Did you try updating pygptj as well? If you are running linux dmesg may give you a clue as to where it is failing
Got the same error on a M1 mac
~/git/privateGPT main ✗ 4h41m ✖ ◒ ⍉
▶ python3.10 privateGPT.py
Using embedded DuckDB with persistence: data will be stored in: db
[1] 72993 illegal hardware instruction python3.10 privateGPT.py
(venv) (base)
PyGPT-J A simple Command Line Interface to test the package Version: 2.0.3
[+] Running model models/ggml-gpt4all-j-v1.3-groovy.bin
gptj_model_load: loading model from 'models/ggml-gpt4all-j-v1.3-groovy.bin' - please wait ...
gptj_model_load: n_vocab = 50400
gptj_model_load: n_ctx = 2048
gptj_model_load: n_embd = 4096
gptj_model_load: n_head = 16
gptj_model_load: n_layer = 28
gptj_model_load: n_rot = 64
gptj_model_load: f16 = 2
[1] 73146 illegal hardware instruction pygptj models/ggml-gpt4all-j-v1.3-groovy.bin
(venv) (base)
dmesg tells me it's this llama cpp library:
python[31999] trap invalid opcode ip:7f8d8fb6559f sp:7ffd0b9559a0 error:0 in _pyllamacpp.cpython-311-x86_64-linux-gnu.so[7f8d8fb55000+5b000]
but again, I've modified its CMakeLists.txt to turn all CPU opts OFF and rebuilt with python -m pip install -e .
I'm using pyenv, which is new to me, so maybe I'm doing something weird there. But I'm in the activated environment the whole time, not issuing any pyenv or related commands. EDIT: this output is the same as the folder I built it in above:
>>> import pkg_resources
>>> pkg_resources.require("pyllamacpp")
[pyllamacpp 1.0.7 (/home/jon/pyllamacpp/pyllamacpp)]
Oh it's definately that. I ran this through gdb: draeician@vivy:~/git/privateGPT$ . venv/bin/activate (venv) draeician@vivy:~/git/privateGPT$ gdb python --->lots of start up message<--- (gdb) run privateGPT.py --->lots of [New Thread 0x7fffd2ff3640 (LWP 129334)] messages<--- [New Thread 0x7ffe92ffd640 (LWP 129612)]
Thread 1 "python" received signal SIGILL, Illegal instruction. 0x00007ffed3723776 in ?? () from /home/draeician/git/privateGPT/venv/lib/python3.10/site-packages/_pyllamacpp.cpython-310-x86_64-linux-gnu.so (gdb) exit A debugging session is active.
Following this guide nomic-ai/pygpt4all#71 If it still fails git clone --recursive https://github.com/abdeladim-s/pygptj.git and install this with pip install e .
Tried both the options but still getting the error. I am running privateGPT in a Manjaro VirtualBox VM, with 4 CPU cores assigned and 10GB of RAM. The Host CPU is Intel Core vPRO i9
I am also Observing : Illegal instruction (core dumped). Note: This is in RHEL container..
+1
Following this guide nomic-ai/pygpt4all#71 If it still fails git clone --recursive https://github.com/abdeladim-s/pygptj.git and install this with pip install e .
Tried both the options but still getting the error. I am running privateGPT in a Manjaro VirtualBox VM, with 4 CPU cores assigned and 10GB of RAM. The Host CPU is Intel Core vPRO i9
Hi all,
I think I have found a solution - more of a workaround. The root cause seems to be Hyper-V is somehow enabled (albeit surreptitiously) on my Windows 10 Host, due to which the VirtualBox Guest (Manjaro Linux) doesn't seem to have access to CPU Instructions like AVX2.
So, I migrated to WSL2 Ubuntu 22.04 and from there, I was able to build this app out of the box. Suits my needs, for the time being.
PS: Not sure why Hyper-V is still "active" despite turning it off explicitly. Also, I was not aware that VirtualBox 7.0.x indeed works with Hyper-V on. With 6.x versions, I vividly remember I was not able to guests with Hyper-V on
Following this guide nomic-ai/pygpt4all#71 If it still fails git clone --recursive https://github.com/abdeladim-s/pygptj.git and install this with pip install e .
Tried both the options but still getting the error. I am running privateGPT in a Manjaro VirtualBox VM, with 4 CPU cores assigned and 10GB of RAM. The Host CPU is Intel Core vPRO i9
Hi all,
I think I have found a solution - more of a workaround. The root cause seems to be Hyper-V is somehow enabled (albeit surreptitiously) on my Windows 10 Host, due to which the VirtualBox Guest (Manjaro Linux) doesn't seem to have access to CPU Instructions like AVX2.
So, I migrated to WSL2 Ubuntu 22.04 and from there, I was able to build this app out of the box. Suits my needs, for the time being.
PS: Not sure why Hyper-V is still "active" despite turning it off explicitly. Also, I was not aware that VirtualBox 7.0.x indeed works with Hyper-V on. With 6.x versions, I vividly remember I was not able to guests with Hyper-V on
Out of curiosity, Hyper-V VM users don't seem to be having such an issue, why are you using vbox? WSL2 also utilises Hyper-V afaik, so I assume you'd have no issues on it.
Following this guide nomic-ai/pygpt4all#71 If it still fails git clone --recursive https://github.com/abdeladim-s/pygptj.git and install this with pip install e .
Tried both the options but still getting the error. I am running privateGPT in a Manjaro VirtualBox VM, with 4 CPU cores assigned and 10GB of RAM. The Host CPU is Intel Core vPRO i9
Hi all, I think I have found a solution - more of a workaround. The root cause seems to be Hyper-V is somehow enabled (albeit surreptitiously) on my Windows 10 Host, due to which the VirtualBox Guest (Manjaro Linux) doesn't seem to have access to CPU Instructions like AVX2. So, I migrated to WSL2 Ubuntu 22.04 and from there, I was able to build this app out of the box. Suits my needs, for the time being. PS: Not sure why Hyper-V is still "active" despite turning it off explicitly. Also, I was not aware that VirtualBox 7.0.x indeed works with Hyper-V on. With 6.x versions, I vividly remember I was not able to guests with Hyper-V on
Out of curiosity, Hyper-V VM users don't seem to be having such an issue, why are you using vbox? WSL2 also utilises Hyper-V afaik, so I assume you'd have no issues on it.
I need to learn the nitty-gritties myself but from whatever I have understood, Microsoft's Hyper-V is a "Type 1" Hypervisor that has all / max control of the underlying hardware. All the guest OS-es don't have as much access / control of the underlying hardware and this could be the reason for advanced CPU Instruction Sets like AVX2 being unavailable to guest OS-es of other apps like VBox. That doesn't seem to be the case for direct guest OS-es of Hyper-V itself.
Before installing VBox, I had ensured that I had disabled Hyper-V completely thru the "Turn Windows features on or off" yet it looks like Hyper-V is/was still running "surreptitiously" - the Turtle icon at the bottom-right of the VBox window indicated that. I abandoned my fight to get rid of that Turtle icon and migrated to WSL2 instead. Btw, I had to use VBox due to my professional work
Following this guide nomic-ai/pygpt4all#71 If it still fails git clone --recursive https://github.com/abdeladim-s/pygptj.git and install this with pip install e .
Tried both the options but still getting the error. I am running privateGPT in a Manjaro VirtualBox VM, with 4 CPU cores assigned and 10GB of RAM. The Host CPU is Intel Core vPRO i9
Hi all, I think I have found a solution - more of a workaround. The root cause seems to be Hyper-V is somehow enabled (albeit surreptitiously) on my Windows 10 Host, due to which the VirtualBox Guest (Manjaro Linux) doesn't seem to have access to CPU Instructions like AVX2. So, I migrated to WSL2 Ubuntu 22.04 and from there, I was able to build this app out of the box. Suits my needs, for the time being. PS: Not sure why Hyper-V is still "active" despite turning it off explicitly. Also, I was not aware that VirtualBox 7.0.x indeed works with Hyper-V on. With 6.x versions, I vividly remember I was not able to guests with Hyper-V on
Out of curiosity, Hyper-V VM users don't seem to be having such an issue, why are you using vbox? WSL2 also utilises Hyper-V afaik, so I assume you'd have no issues on it.
I need to learn the nitty-gritties myself but from whatever I have understood, Microsoft's Hyper-V is a "Type 1" Hypervisor that has all / max control of the underlying hardware. All the guest OS-es don't have as much access / control of the underlying hardware and this could be the reason for advanced CPU Instruction Sets like AVX2 being unavailable to guest OS-es of other apps like VBox. That doesn't seem to be the case for direct guest OS-es of Hyper-V itself.
Before installing VBox, I had ensured that I had disabled Hyper-V completely thru the "Turn Windows features on or off" yet it looks like Hyper-V is/was still running "surreptitiously" - the Turtle icon at the bottom-right of the VBox window indicated that. I abandoned my fight to get rid of that Turtle icon and migrated to WSL2 instead. Btw, I had to use VBox due to my professional work
alright, then you can try this: https://stackoverflow.com/questions/65780506/how-to-enable-avx-avx2-in-virtualbox-6-1-16-with-ubuntu-20-04-64bit or disable the windows antivirus completely, as far as i know that's what causes the instruction blockage.