private-gpt icon indicating copy to clipboard operation
private-gpt copied to clipboard
verified publisher icon zylon-ai

GGML_ASSERT: /project/ggml/src/ggml.c:3732: ctx->mem_buffer != NULL Aborted

Open hanwsf opened this issue 2 years ago • 3 comments

python privateGPT.py llama.cpp: loading model from models/ggml-model-q4_0.bin llama.cpp: can't use mmap because tensors are not aligned; convert to new format to avoid this llama_model_load_internal: format = 'ggml' (old version with low tokenizer quality and no mmap support) llama_model_load_internal: n_vocab = 32000 llama_model_load_internal: n_ctx = 1000 llama_model_load_internal: n_embd = 4096 llama_model_load_internal: n_mult = 256 llama_model_load_internal: n_head = 32 llama_model_load_internal: n_layer = 32 llama_model_load_internal: n_rot = 128 llama_model_load_internal: ftype = 2 (mostly Q4_0) llama_model_load_internal: n_ff = 11008 llama_model_load_internal: n_parts = 1 llama_model_load_internal: model size = 7B llama_model_load_internal: ggml ctx size = 4113748.20 KB llama_model_load_internal: mem required = 5809.33 MB (+ 2052.00 MB per state) ................................................................................................... . llama_init_from_file: kv self size = 1000.00 MB AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 | Using embedded DuckDB with persistence: data will be stored in: db gptj_model_load: loading model from 'models/ggml-gpt4all-j-v1.3-groovy.bin' - please wait ... gptj_model_load: n_vocab = 50400 gptj_model_load: n_ctx = 2048 gptj_model_load: n_embd = 4096 gptj_model_load: n_head = 16 gptj_model_load: n_layer = 28 gptj_model_load: n_rot = 64 gptj_model_load: f16 = 2 gptj_model_load: ggml ctx size = 4505.45 MB GGML_ASSERT: /project/ggml/src/ggml.c:3732: ctx->mem_buffer != NULL Aborted

Out of Ram? Thanks!

hanwsf avatar May 13 '23 13:05 hanwsf

Ubuntu18, gcc-11

hanwsf avatar May 14 '23 10:05 hanwsf

I have the same problem, did you solve it please?

Yiran-young avatar May 14 '23 12:05 Yiran-young

Not yet. Use ubuntu:latest docker image, also see the same error: llama_init_from_file: kv self size = 1000.00 MB AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 | Using embedded DuckDB with persistence: data will be stored in: db gptj_model_load: loading model from 'models/ggml-gpt4all-j-v1.3-groovy.bin' - please wait ... gptj_model_load: n_vocab = 50400 gptj_model_load: n_ctx = 2048 gptj_model_load: n_embd = 4096 gptj_model_load: n_head = 16 gptj_model_load: n_layer = 28 gptj_model_load: n_rot = 64 gptj_model_load: f16 = 2 gptj_model_load: ggml ctx size = 4505.45 MB GGML_ASSERT: /project/ggml/src/ggml.c:3732: ctx->mem_buffer != NULL Aborted

hanwsf avatar May 14 '23 13:05 hanwsf

I also had the same error. Making some space in my storage by deleting some files/programs fixed the issue for me.

bluuo avatar May 15 '23 09:05 bluuo

I also had the same error. Making some space in my storage by deleting some files/programs fixed the issue for me.

how much space left on hard drive can make this work?

hanwsf avatar May 15 '23 13:05 hanwsf

short of memory. Needs 16GB ram (used around 13GB).

hanwsf avatar May 15 '23 14:05 hanwsf

i have 246gb on the hard drive and 32gb of ram

pbm94 avatar May 17 '23 16:05 pbm94

have the same issue

raikrahul avatar May 23 '23 02:05 raikrahul

i am using system with 16 GB RAM and there is 26GB free in HDD. still facing the same issue.. can anyone suggest how to resolve?

ADXXI1590 avatar May 26 '23 05:05 ADXXI1590

I managed to resolve the issue after increase the memory to 16GB.

In case you are like me, running Ubuntu under WSL, and trying to test privateGPT.py. In your Ubuntu VM, run 'free -h' to check your RAM size. Should be at least 16GB, like below:

─$ free -h
               total        used        free      shared  buff/cache   available
Mem:            15Gi       385Mi        14Gi       3.0Mi       447Mi        14Gi
Swap:          4.0Gi          0B       4.0Gi

If not, check your WSL config, and allocate 16GB RAM to your Ubuntu.

  • At your Windows cmd prompt, such as c:\users\XXXX, create a file ".wslconfig" if it doesn't exist.
  • Paste the following:
[wsl2]
memory=16GB
processors=4
  • then restart your WSL, with "wsl --shutdown", and start your Ubuntu instance again.
  • Run the cmd free -h and make sure you have 16GB ram.

myseq avatar Jun 04 '23 17:06 myseq

I'm facing the same problem. Does it mean I have to have 16GiB FREE space RAM for running PrivateGPT?

PS> python .\privateGPT.py
Using embedded DuckDB with persistence: data will be stored in: db
Found model file.
gptj_model_load: loading model from 'models/ggml-gpt4all-j-v1.3-groovy.bin' - please wait ...
gptj_model_load: n_vocab = 50400
gptj_model_load: n_ctx   = 2048
gptj_model_load: n_embd  = 4096
gptj_model_load: n_head  = 16
gptj_model_load: n_layer = 28
gptj_model_load: n_rot   = 64
gptj_model_load: f16     = 2
gptj_model_load: ggml ctx size = 5401.45 MB
GGML_ASSERT: C:\Users\circleci.PACKER-64370BA5\project\gpt4all-backend\llama.cpp\ggml.c:4411: ctx->mem_buffer != NULL

System Info

  • Microsoft Windows 11 Professional 22H2
  • Python for Windows v3.11.3
  • 16GiB physical RAM
    • 10.4GiB (386MiB) used
    • 5.3GiB available
    • 14.9/18.3GiB commited
    • 5.4GiB cached
    • 944MiB page cache pool
    • 666MiB non-page cache pool
  • 1TB physical SSD
    • C:\ 247GiB total with 108GiB available
    • D:\ 692GiB total with 600GiB available

Dragon1573 avatar Jun 08 '23 02:06 Dragon1573

windows 16GB is not sufficient. Add swapper more, then it might work, but very slow

---- 回复的原邮件 ---- | 发件人 | @.> | | 日期 | 2023年06月08日 10:41 | | 收件人 | @.> | | 抄送至 | @.>、State @.> | | 主题 | Re: [imartinez/privateGPT] GGML_ASSERT: /project/ggml/src/ggml.c:3732: ctx->mem_buffer != NULL Aborted (Issue #90) |

I'm facing the same problem. Does it mean I have to have 16GiB FREE space RAM for running PrivateGPT?

PS> python .\privateGPT.py Using embedded DuckDB with persistence: data will be stored in: db Found model file. gptj_model_load: loading model from 'models/ggml-gpt4all-j-v1.3-groovy.bin' - please wait ... gptj_model_load: n_vocab = 50400 gptj_model_load: n_ctx = 2048 gptj_model_load: n_embd = 4096 gptj_model_load: n_head = 16 gptj_model_load: n_layer = 28 gptj_model_load: n_rot = 64 gptj_model_load: f16 = 2 gptj_model_load: ggml ctx size = 5401.45 MB GGML_ASSERT: C:\Users\circleci.PACKER-64370BA5\project\gpt4all-backend\llama.cpp\ggml.c:4411: ctx->mem_buffer != NULL

System Info Microsoft Windows 11 Professional 22H2 Python for Windows v3.11.3 16GiB physical RAM 10.4GiB (386MiB) used 5.3GiB available 14.9/18.3GiB commited 5.4GiB cached 944MiB page cache pool 666MiB non-page cache pool 1TB physical SSD C:\ 247GiB total with 108GiB available D:\ 692GiB total with 600GiB available

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you modified the open/close state.Message ID: @.***>

hanwsf avatar Jun 08 '23 12:06 hanwsf

RTX 4090 24B video memory 64GB DDR5 i9 - 14th gen and still have this error: GGML_ASSERT: /tmp/tmp25xydfjh/llama_cpp_python-0.2.53/vendor/llama.cpp/ggml-cuda.cu:8620: ptr == (void *)(g_cuda_pool_addr[device] + g_cuda_pool_used[device]) make: *** [Makefile:36: run] Aborted

iulix21 avatar Jun 03 '24 12:06 iulix21