Optimize `dmh` speed - it's very slow right now.

[x0urc3]

Questions	Answers
OS/arch/bits (mandatory)	Ubuntu x86
File format of the file you reverse (mandatory)	ELF
Architecture/bits of the file (mandatory)	x86/64
r2 -v full output, not truncated (mandatory)	radare2 4.2.0-git 23640 @ linux-x86-64 git.4.1.1-153-gd0a915458 commit: d0a915458abaa429926840c9fa41e2cd1fedfb1b build: 2020-01-22__11:37:15

Expected behavior

dmh produce result in reasonable amout of time

Actual behavior

dmh is slow even though there's only one chunk in the heap

Steps to reproduce the behavior

$ r2 -d test/bins/elf/simple_malloc_x86_64 
Process with PID 15946 started...
= attach 15946 15946
bin.baddr 0x55b6ab962000
Using 0x55b6ab962000
asm.bits 64
Warning: r_bin_file_hash: file exceeds bin.hashlimit
 -- This shell has been seized by the Internet's Police.
[0x7f6041d2d100]> dcu main
Continue until 0x55b6ab96270a using 1 bpsize
hit breakpoint at: 55b6ab96270a
[0x55b6ab96270a]> ?t dmh
10.411503

  Malloc chunk @ 0x55b6ad0e8290 [size: 0x11c10][allocated]
  Top chunk @ 0x55b6ad0f9ea0 - [brk_start: 0x55b6ad0e8000, brk_end: 0x55b6ad109000]

[0x55b6ab96270a]> 

[radare]

didnt tried but i guess those 10s are spent parsing the libc library and loading all the symbol information and metadata into r2

[x0urc3]

didnt tried but i guess those 10s are spent parsing the libc library and loading all the symbol information and metadata into r2

Your right. Brute force search to find main_arena. Any other method to find main_arena address in memory?

https://github.com/radareorg/radare2/blob/14977a6828b47ee5a70e6c41c6d5dce873b78d48/libr/core/linux_heap_glibc.c#L343-L353

[meowmeowxw]

pwndbg checks main_arena in the symbols of the libc: https://github.com/pwndbg/pwndbg/blob/dev/pwndbg/heap/ptmalloc.py#L73

while gef compute the offset from __malloc_hook since main_arena and __malloc_hook are adjacient in memory: https://github.com/hugsy/gef/blob/dev/gef.py#L58

It shouldn't be too hard to implement one of the two methods, I can try in the following days

[radare]

Cool!

On 14 Apr 2020, at 21:55, meowmeowxw [email protected] wrote:

 pwndbg checks main_arena in the symbols of the libc: https://github.com/pwndbg/pwndbg/blob/dev/pwndbg/heap/ptmalloc.py#L73

while gef compute the offset from __malloc_hook since main_arena and __malloc_hook are adjacient in memory: https://github.com/hugsy/gef/blob/dev/gef.py#L58

It shouldn't be too hard to implement one of the two methods, I can try in the following days

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[meowmeowxw]

Check new implementation in the updated comment. ~~I've added the way to compute the main_arena using symbols:~~ ~~https://github.com/meowmeowxw/radare2/blob/cab8f2a92f5ee7de7a7fd00165f2f01c3ebff8dd/libr/core/linux_heap_glibc.c#L325~~

However if I call r_core_free() to free the core of the libc, radare2 crashes a (segfault after executing dmh) because of the call to r_core_task_scheduler_fini() inside r_core_fini(), I don't know exactly how to bypass that call (without copying the code of r_core_fini) or why r_core_task_scheduler_fini() crashes the program. I would like a little help on this part since this is my first issue :).

// this is with r_core_free (libc_core);
[0x7ffff7fd3100]> db 0x55555555525a
[0x7ffff7fd3100]> dc
hit breakpoint at: 55555555525a
[0x55555555525a]> dmht
1833440
time with sym: 0.097662
Tcache main arena @ 0x7ffff7f5e9e0
bin : 0, items : 7, fd :0x555555559350->0x555555559330->0x555555559310->0x5555555592f0->0x5555555592d0->0x5555555592b0->0x555555559290
[0x55555555525a]> 
[1]    16741 segmentation fault (core dumped)  ./radare2 -d ~/Security/test/heap/tcache

I measured the time to executes the function r_resolve_main_arena and now it takes roughly 0.1 second on my machine, while the bruteforce takes around 1/1.5 seconds.

The weird thing is that if I call dmh before dmht then it takes lots of time (from 15 to 30 seconds), however the call to r_resolve_main_arena is still executed with 0.1 seconds (with my patch). However if call dmht or dmhf before dmh, then dmh is executed immediately, so the problem is not in r_resolve_main_arena.

[meowmeowxw]

UPDATE:

I used a function to resolve main_arena from libc:

https://github.com/meowmeowxw/radare2/blob/0c41fd4ebd2c33c637efad582db4185f3ae984a4/libr/core/linux_heap_glibc.c#L24

which is called by: https://github.com/meowmeowxw/radare2/blob/0c41fd4ebd2c33c637efad582db4185f3ae984a4/libr/core/linux_heap_glibc.c#L363

I created a source test and the binary of the test.

Then I created a script to launch with radare2.

Finally I compiled the program with sys/sanitize.sh

Now running:

$ export ASAN_OPTIONS=detect_odr_violation=0
$ ./radare2 -i script.rr2 -d ../../test/tcache #./radare2 --> ./binr/radare2/radare2

yields to:

./radare2 -i script.rr2 -d ../../test/tcache
Process with PID 92463 started...
= attach 92463 92463
bin.baddr 0x00400000
Using 0x400000
asm.bits 64
[x] Analyze all flags starting with sym. and entry0 (aa)
[x] Analyze function calls (aac)
[x] Analyze len bytes of instructions for references (aar)
[x] Check for objc references
[x] Check for vtables
[TOFIX: aaft can't run in debugger mode.ions (aaft)
[x] Type matching analysis for all functions (aaft)
[x] Propagate noreturn information
[x] Use -AA or aaaa to perform additional experimental analysis.
chunk: 0        address: 0x101c2a0      value: 0
chunk: 1        address: 0x101c2c0      value: 1
chunk: 2        address: 0x101c2e0      value: 2
chunk: 3        address: 0x101c300      value: 3
chunk: 4        address: 0x101c320      value: 4
chunk: 5        address: 0x101c340      value: 5
chunk: 6        address: 0x101c360      value: 6
chunk: 7        address: 0x101c380      value: 7
chunk: 8        address: 0x101c3a0      value: 8
chunk: 9        address: 0x101c3c0      value: 9
hit breakpoint at: 401221
TRYING
FOUND: 0x7f7ea69219e0
time with sym: 0.378699
Tcache main arena @ 0x7f7ea69219e0
bin : 0, items : 7, fd :0x101c350->0x101c330->0x101c310->0x101c2f0->0x101c2d0->0x101c2b0->0x101c290                                                                                                                                                                                        
 -- I accidentally the kernel with radare2.
[0x00401221]> =================================================================
==92461==ERROR: AddressSanitizer: heap-use-after-free on address 0x7f0d83aa07c8 at pc 0x7f0d9647f995 bp 0x7ffc85f25390 sp 0x7ffc85f25380
READ of size 8 at 0x7f0d83aa07c8 thread T0
    #0 0x7f0d9647f994 in r_core_task_self /home/meowmeow/Tools/radare2-dmh/libr/core/task.c:552
    #1 0x7f0d95ee9c2e in r_core_sleep_begin /home/meowmeow/Tools/radare2-dmh/libr/core/core.c:2314
    #2 0x7f0d9c67dbae in r_cons_sleep_begin /home/meowmeow/Tools/radare2-dmh/libr/cons/cons.c:419
    #3 0x7f0d9c6b92c5 in r_cons_readchar /home/meowmeow/Tools/radare2-dmh/libr/cons/input.c:614
    #4 0x7f0d9c69ce42 in r_line_readchar_utf8 /home/meowmeow/Tools/radare2-dmh/libr/cons/dietline.c:221
    #5 0x7f0d9c6a9bd3 in r_line_readline_cb /home/meowmeow/Tools/radare2-dmh/libr/cons/dietline.c:1303
    #6 0x7f0d9c6a5b44 in r_line_readline /home/meowmeow/Tools/radare2-dmh/libr/cons/dietline.c:836
    #7 0x7f0d95ee4969 in r_core_fgets /home/meowmeow/Tools/radare2-dmh/libr/core/core.c:1953
    #8 0x7f0d9c6b86ac in r_cons_fgets /home/meowmeow/Tools/radare2-dmh/libr/cons/input.c:365
    #9 0x7f0d95ef815d in r_core_prompt /home/meowmeow/Tools/radare2-dmh/libr/core/core.c:2972
    #10 0x7f0d95ef5fde in r_core_prompt_loop /home/meowmeow/Tools/radare2-dmh/libr/core/core.c:2839
    #11 0x7f0d9d780759 in r_main_radare2 /home/meowmeow/Tools/radare2-dmh/libr/main/radare2.c:1365
    #12 0x5566c7a0f713 in main /home/meowmeow/Tools/radare2-dmh/binr/radare2/radare2.c:96
    #13 0x7f0d9cbd3001 in __libc_start_main (/usr/lib/libc.so.6+0x27001)
    #14 0x5566c7a0f19d in _start (/home/meowmeow/Tools/radare2-dmh/binr/radare2/radare2+0x219d)

0x7f0d83aa07c8 is located 397256 bytes inside of 397504-byte region [0x7f0d83a3f800,0x7f0d83aa08c0)
freed by thread T0 here:
    #0 0x7f0d9e42e0e9 in __interceptor_free /build/gcc/src/gcc/libsanitizer/asan/asan_malloc_linux.cpp:123
    #1 0x7f0d95ef5fbd in r_core_free /home/meowmeow/Tools/radare2-dmh/libr/core/core.c:2832
    #2 0x7f0d95f4b211 in get_va_symbol_64 /home/meowmeow/Tools/radare2-dmh/libr/core/linux_heap_glibc.c:49
    #3 0x7f0d95f53bb5 in r_resolve_main_arena_64 /home/meowmeow/Tools/radare2-dmh/libr/core/linux_heap_glibc.c:363
    #4 0x7f0d95f6604f in cmd_dbg_map_heap_glibc_64 /home/meowmeow/Tools/radare2-dmh/libr/core/linux_heap_glibc.c:1583
    #5 0x7f0d95f29bdb in r_debug_heap /home/meowmeow/Tools/radare2-dmh/libr/core/cmd_debug.c:1548
    #6 0x7f0d95f2ec75 in cmd_debug_map /home/meowmeow/Tools/radare2-dmh/libr/core/cmd_debug.c:1919
    #7 0x7f0d95f9b0d8 in cmd_debug /home/meowmeow/Tools/radare2-dmh/libr/core/cmd_debug.c:5130
    #8 0x7f0d96345c6d in r_cmd_call /home/meowmeow/Tools/radare2-dmh/libr/core/cmd_api.c:291
    #9 0x7f0d961ae2d8 in r_core_cmd_subst_i /home/meowmeow/Tools/radare2-dmh/libr/core/cmd.c:3772
    #10 0x7f0d9619eb5b in r_core_cmd_subst /home/meowmeow/Tools/radare2-dmh/libr/core/cmd.c:2690
    #11 0x7f0d9619f1df in r_core_cmd_subst /home/meowmeow/Tools/radare2-dmh/libr/core/cmd.c:2722
    #12 0x7f0d9619f1df in r_core_cmd_subst /home/meowmeow/Tools/radare2-dmh/libr/core/cmd.c:2722
    #13 0x7f0d9619f1df in r_core_cmd_subst /home/meowmeow/Tools/radare2-dmh/libr/core/cmd.c:2722
    #14 0x7f0d961e10c8 in run_cmd_depth /home/meowmeow/Tools/radare2-dmh/libr/core/cmd.c:6669
    #15 0x7f0d961e1fd4 in r_core_cmd /home/meowmeow/Tools/radare2-dmh/libr/core/cmd.c:6745
    #16 0x7f0d961e2186 in r_core_cmd_lines /home/meowmeow/Tools/radare2-dmh/libr/core/cmd.c:6777
    #17 0x7f0d961e2568 in r_core_cmd_file /home/meowmeow/Tools/radare2-dmh/libr/core/cmd.c:6817
    #18 0x7f0d96190317 in r_core_run_script /home/meowmeow/Tools/radare2-dmh/libr/core/cmd.c:1166
    #19 0x7f0d9d773d94 in run_commands /home/meowmeow/Tools/radare2-dmh/libr/main/radare2.c:249
    #20 0x7f0d9d77fa82 in r_main_radare2 /home/meowmeow/Tools/radare2-dmh/libr/main/radare2.c:1312
    #21 0x5566c7a0f713 in main /home/meowmeow/Tools/radare2-dmh/binr/radare2/radare2.c:96
    #22 0x7f0d9cbd3001 in __libc_start_main (/usr/lib/libc.so.6+0x27001)

previously allocated by thread T0 here:
    #0 0x7f0d9e42e639 in __interceptor_calloc /build/gcc/src/gcc/libsanitizer/asan/asan_malloc_linux.cpp:154
    #1 0x7f0d95ed3ae7 in r_core_new /home/meowmeow/Tools/radare2-dmh/libr/core/core.c:827
    #2 0x7f0d95f4ae24 in get_va_symbol_64 /home/meowmeow/Tools/radare2-dmh/libr/core/linux_heap_glibc.c:27
    #3 0x7f0d95f53bb5 in r_resolve_main_arena_64 /home/meowmeow/Tools/radare2-dmh/libr/core/linux_heap_glibc.c:363
    #4 0x7f0d95f6604f in cmd_dbg_map_heap_glibc_64 /home/meowmeow/Tools/radare2-dmh/libr/core/linux_heap_glibc.c:1583
    #5 0x7f0d95f29bdb in r_debug_heap /home/meowmeow/Tools/radare2-dmh/libr/core/cmd_debug.c:1548
    #6 0x7f0d95f2ec75 in cmd_debug_map /home/meowmeow/Tools/radare2-dmh/libr/core/cmd_debug.c:1919
    #7 0x7f0d95f9b0d8 in cmd_debug /home/meowmeow/Tools/radare2-dmh/libr/core/cmd_debug.c:5130
    #8 0x7f0d96345c6d in r_cmd_call /home/meowmeow/Tools/radare2-dmh/libr/core/cmd_api.c:291
    #9 0x7f0d961ae2d8 in r_core_cmd_subst_i /home/meowmeow/Tools/radare2-dmh/libr/core/cmd.c:3772
    #10 0x7f0d9619eb5b in r_core_cmd_subst /home/meowmeow/Tools/radare2-dmh/libr/core/cmd.c:2690
    #11 0x7f0d9619f1df in r_core_cmd_subst /home/meowmeow/Tools/radare2-dmh/libr/core/cmd.c:2722
    #12 0x7f0d9619f1df in r_core_cmd_subst /home/meowmeow/Tools/radare2-dmh/libr/core/cmd.c:2722
    #13 0x7f0d9619f1df in r_core_cmd_subst /home/meowmeow/Tools/radare2-dmh/libr/core/cmd.c:2722
    #14 0x7f0d961e10c8 in run_cmd_depth /home/meowmeow/Tools/radare2-dmh/libr/core/cmd.c:6669
    #15 0x7f0d961e1fd4 in r_core_cmd /home/meowmeow/Tools/radare2-dmh/libr/core/cmd.c:6745
    #16 0x7f0d961e2186 in r_core_cmd_lines /home/meowmeow/Tools/radare2-dmh/libr/core/cmd.c:6777
    #17 0x7f0d961e2568 in r_core_cmd_file /home/meowmeow/Tools/radare2-dmh/libr/core/cmd.c:6817
    #18 0x7f0d96190317 in r_core_run_script /home/meowmeow/Tools/radare2-dmh/libr/core/cmd.c:1166
    #19 0x7f0d9d773d94 in run_commands /home/meowmeow/Tools/radare2-dmh/libr/main/radare2.c:249
    #20 0x7f0d9d77fa82 in r_main_radare2 /home/meowmeow/Tools/radare2-dmh/libr/main/radare2.c:1312
    #21 0x5566c7a0f713 in main /home/meowmeow/Tools/radare2-dmh/binr/radare2/radare2.c:96
    #22 0x7f0d9cbd3001 in __libc_start_main (/usr/lib/libc.so.6+0x27001)

SUMMARY: AddressSanitizer: heap-use-after-free /home/meowmeow/Tools/radare2-dmh/libr/core/task.c:552 in r_core_task_self
Shadow bytes around the buggy address:
  0x0fe23074c0a0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0fe23074c0b0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0fe23074c0c0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0fe23074c0d0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0fe23074c0e0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
=>0x0fe23074c0f0: fd fd fd fd fd fd fd fd fd[fd]fd fd fd fd fd fd
  0x0fe23074c100: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0fe23074c110: fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa
  0x0fe23074c120: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0fe23074c130: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0fe23074c140: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==92461==ABORTING


chunk: 0        address: 0x101c7f0      value: 0
chunk: 1        address: 0x101c3a0      value: 1
chunk: 2        address: 0x101c380      value: 2
chunk: 3        address: 0x101c3c0      value: 3
chunk: 4        address: 0x101c2a0      value: 4
chunk: 5        address: 0x101c2c0      value: 5
chunk: 6        address: 0x101c2e0      value: 6
chunk: 7        address: 0x101c300      value: 7
chunk: 8        address: 0x101c320      value: 8
chunk: 9        address: 0x101c340      value: 9

So yeah that's something going on with the task_scheduler. I found two ugly solution to solve the issue:

1. Never free the core of the libc, which is a bad practice
2. Re-implement r_core_fini() without the call to r_core_task_scheduler_fini().

Any idea on how to solve the problem in a proper way?

[ret2libc]

cc @thestr4ng3r

[thestr4ng3r]

Well yeah unfortunately that is the expected behavior as long as there is the global RCons singleton. Using multiple RCores in a single process will lead to these issues. Could you maybe use only RBin? Maybe it needs some refactoring to be independent enough to work there but it would be the more elegant solution.

[trufae]

Can you retry? there are many statements in this issue that are no longer true and im just catching up with all the issues from the dramatic 2020