heap buffer overflow is expected

[yuweol]

Hello.

While i'm writing a code with mpc.c, address sanitizer reported a heap buffer overflow. I attached a file. (both bug1.c and bug2.c can cause heap buffer overflow. please test them separately.)

commit number : 3c266180c4984f1d5aba35e4e3ecc5f4776bc398

And the build command I used is below.

clang -fsanitize=address -m32 -g -ansi -pedantic -O3 -Wall -Wextra -Wformat=2 -Wshadow -Wno-long-long -Wno-overlength-strings -Wno-format-nonliteral -Wcast-align -Wwrite-strings -Wstrict-prototypes -Wold-style-definition -Wredundant-decls -Wnested-externs -Wmissing-include-dirs -Wswitch-default -o bug1 bug1.c ./mpc.c && ./bug1

clang -fsanitize=address -m32 -g -ansi -pedantic -O3 -Wall -Wextra -Wformat=2 -Wshadow -Wno-long-long -Wno-overlength-strings -Wno-format-nonliteral -Wcast-align -Wwrite-strings -Wstrict-prototypes -Wold-style-definition -Wredundant-decls -Wnested-externs -Wmissing-include-dirs -Wswitch-default -o bug2 bug2.c ./mpc.c && ./bug2

It may my problem because I didn't fully understand this code, but anyway please check this.

bug.zip

[orangeduck]

Hi yuweol,

Thanks for the report. There could be a memory bug in mpc however I can see some bugs in your code which are probably worth fixing first:

In bug1.c you don't free the result of mpc_parse which is stored in r. Either it will be a mpc_err_t or a mpc_ast_t you need to check the return code of mpc_parse to see. You can see an example of this in the code in the quickstart: https://github.com/orangeduck/mpc#quickstart . Also you don't ever free the string bug1 you allocate.

In bug2.c you don't free any of the parsers you allocate - this needs to be done with mpc_cleanup(4, Expr, Prod, Value, Maths); like in the quickstart example again.

Finally, I also sometimes get heap memory errors when using mpc which come from inside functions such as strcmp and strcpy. The reason for this is that sometimes when these functions are compiled using optimizations they use SIMD instructions which read or write past the end of the allocation. It depends a bit on the exact nature of the report but in most cases these heap memory errors can be ignored because these optimizations are performed in a safe way (by making some basic assumptions about allocation sizes etc).

If you attach the full report/output I can take a closer look.

Hope that helps,

Dan

[yuweol]

Hello, Thanks for your reply. My 2 code files are just examples for reproducing bugs inside mpc. Below is bug report of address sanitizer, and at this time I compiled the code with option -O0 (without any optimization)

<bug1.c>

==6458==ERROR: AddressSanitizer: heap-buffer-overflow on address 0xf51005ef at pc 0x0817cc32 bp 0xfffac958 sp 0xfffac94c READ of size 1 at 0xf51005ef thread T0 #0 0x817cc31 in mpcf_strtrimr mpc.c:2135:10 #1 0x816a5f2 in mpc_parse_input mpc.c:1042:13 #2 0x81743ce in mpc_parse mpc.c:1197:7 #3 0x81654a3 in main bug1.c:12:3 #4 0xf7495285 in __libc_start_main (/lib/i386-linux-gnu/libc.so.6+0x18285) #5 0x8064237 in _start (bug1+0x8064237)

0xf51005ef is located 1 bytes to the left of 1-byte region [0xf51005f0,0xf51005f1) allocated by thread T0 here: #0 0x812a824 in calloc (bug1+0x812a824) #1 0x817978e in mpcf_strfold mpc.c:2305:13 #2 0x81737d1 in mpc_stack_merger_out mpc.c:939:18 #3 0x816c014 in mpc_parse_input mpc.c:1105:13 #4 0x81743ce in mpc_parse mpc.c:1197:7 #5 0x81654a3 in main bug1.c:12:3 #6 0xf7495285 in __libc_start_main (/lib/i386-linux-gnu/libc.so.6+0x18285)

SUMMARY: AddressSanitizer: heap-buffer-overflow mpc.c:2135:10 in mpcf_strtrimr Shadow bytes around the buggy address: 0x3ea20060: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x3ea20070: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x3ea20080: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x3ea20090: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x3ea200a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa =>0x3ea200b0: fa fa fa fa fa fa fa fa fa fa fa fa fa[fa]01 fa 0x3ea200c0: fa fa 00 06 fa fa 04 fa fa fa 07 fa fa fa fd fd 0x3ea200d0: fa fa fd fa fa fa fd fa fa fa fd fd fa fa fd fa 0x3ea200e0: fa fa fd fd fa fa fd fa fa fa 01 fa fa fa 07 fa 0x3ea200f0: fa fa 00 06 fa fa 00 07 fa fa fa fa fa fa fa fa 0x3ea20100: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ==6458==ABORTING

<bug2.c>

==7438==ERROR: AddressSanitizer: heap-buffer-overflow on address 0xf5020314 at pc 0x08129b01 bp 0xff9e9968 sp 0xff9e9540 READ of size 8 at 0xf5020314 thread T0 #0 0x8129b00 in __asan_memmove (bug2+0x8129b00) #1 0x818b409 in mpc_optimise_unretained mpc.c:3426:7 #2 0x8188d23 in mpc_optimise_unretained mpc.c:3358:7 #3 0x817c916 in mpc_optimise mpc.c:3505:3 #4 0x8194d04 in mpca_stmt_list_apply_to mpc.c:3109:5 #5 0x816a9c4 in mpc_parse_input mpc.c:1052:13 #6 0x8186894 in mpca_lang_st mpc.c:3180:8 #7 0x8186f68 in mpca_lang mpc.c:3250:9 #8 0x81653e6 in main bug2.c:12:3 #9 0xf7575285 in __libc_start_main (/lib/i386-linux-gnu/libc.so.6+0x18285) #10 0x8064237 in _start (bug2+0x8064237)

0xf5020314 is located 0 bytes to the right of 4-byte region [0xf5020310,0xf5020314) allocated by thread T0 here: #0 0x812a624 in malloc (bug2+0x812a624) #1 0x8184013 in mpca_and mpc.c:2780:21 #2 0x8184f74 in mpcaf_grammar_and mpc.c:2857:30 #3 0x81736b1 in mpc_stack_merger_out mpc.c:939:18 #4 0x816c674 in mpc_parse_input mpc.c:1121:15 #5 0x8186894 in mpca_lang_st mpc.c:3180:8 #6 0x8186f68 in mpca_lang mpc.c:3250:9 #7 0x81653e6 in main bug2.c:12:3 #8 0xf7575285 in __libc_start_main (/lib/i386-linux-gnu/libc.so.6+0x18285)

SUMMARY: AddressSanitizer: heap-buffer-overflow (bug2+0x8129b00) in __asan_memmove Shadow bytes around the buggy address: 0x3ea04010: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa 0x3ea04020: fa fa fd fa fa fa fd fd fa fa fd fa fa fa fd fd 0x3ea04030: fa fa fd fa fa fa fd fa fa fa fd fd fa fa fd fa 0x3ea04040: fa fa fd fd fa fa fd fa fa fa fd fa fa fa fd fd 0x3ea04050: fa fa fd fd fa fa fd fd fa fa fd fa fa fa fd fa =>0x3ea04060: fa fa[04]fa fa fa 00 fa fa fa fd fa fa fa fd fa 0x3ea04070: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa 0x3ea04080: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa 0x3ea04090: fa fa fd fd fa fa fd fa fa fa fd fd fa fa fd fa 0x3ea040a0: fa fa fd fa fa fa fd fd fa fa fd fa fa fa fd fd 0x3ea040b0: fa fa fd fa fa fa fd fa fa fa fd fd fa fa fd fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ==7438==ABORTING

As you can see, address sanitizer found a bug and forcefully terminated a program while each mpc_parse, mpca_lang function was working. So I don't think this bugs are because of my misuse of mpc.

Thanks.

[orangeduck]

Thanks for the full report I will look into it a bit more.

[orangeduck]

I wasn't able to reproduce these crashes. I think I need to use valgrind to get more information on this which means installing Ubuntu however I am still surprised you are getting errors in this case. These sort of examples have been tried and tested many times...