dynamorio
dynamorio copied to clipboard
DynamoRIO
recreate_app_state_from_ilist instrs mismatch with L0_filter_until_instrs
If the burst_gencode test is run with -L0_filter_until_instrs X by setting it in the options at https://github.com/DynamoRIO/dynamorio/blob/34fbc25899c45727949332a43050d746aafc83d8/clients/drcachesim/tests/burst_gencode.cpp#L302, it hits the following assert while handling the ud2 (which the test app makes on purpose to test signals in gencode).
SYSLOG_ERROR: Application dynamorio/suite/tests/bin/tool.drcacheoff.gencode (1415995). Internal Error: DynamoRIO debug check failure: /usr/local/google/home/sharmaabhinav/dr/src/i6466-3/core/unix/signal.c:3131 false
The proximal cause seems to be hitting this case during recreate_app_state_from_ilist: https://github.com/DynamoRIO/dynamorio/blob/34fbc25899c45727949332a43050d746aafc83d8/core/translate.c#L1027
This happens when, while walking the ilist looking for the faulting instr, we overshoot the target_cache pc. I printed out the instr disassembly at each iteration for the ilist instr and the instr at cpc, which should both match. Looks like they start mismatching very early on:
instrs in the given ilist:
ilist for recreation:
TAG 0x00007f3128700000
+0 m4 @0x00007f2ee81f4af0 48 a1 48 f4 a5 49 19 mov <rel> 0x0000561949a5f448[8byte] -> %rax
56 00 00
+10 m4 @0x00007f2ee81f0a18 <label note=0x0000000000000008>
+10 m4 @0x00007f2ee81f3ff8 48 3d 03 00 00 00 cmp %rax $0x0000000000000003
+16 m4 @0x00007f2ee81f13c8 0f 85 fa ff ff ff jnz @0x00007f2ee81f3998[8byte]
+22 L3 @0x00007f2ee81f0b28 90 nop
+23 L3 @0x00007f2ee81f4818 b8 42 07 00 00 mov $0x00000742 -> %eax
+28 L3 @0x00007f2ee81f4570 b8 27 04 00 00 mov $0x00000427 -> %eax
...
instrs manually disassembled at cpc:
cpc 5619093cb73c target_cache 5619093cb869
mov <rel> 0x0000561949a5f448[8byte] -> %rax # corresponds to mov <rel> 0x0000561949a5f448[8byte] -> %rax in the ilist
bytes: 48 8b 05 05 3d 69 40
cpc 5619093cb746 target_cache 5619093cb869
add %al (%rax)[1byte] -> (%rax)[1byte] # corresponds to cmp %rax $0x0000000000000003 in the ilist
bytes:00 00
cpc 5619093cb74c target_cache 5619093cb869
add %al (%rax)[1byte] -> (%rax)[1byte] # corresponds to jnz @0x00007f2ee81f3998[8byte] in the ilist
bytes:00 00
The very first mov instr in the ilist is 10 byte, but when it is disassembled from cpc, it seems to be only 7 bytes. This then mis-aligns all later instrs.
Surprising that DR disassembles both as mov <rel> 0x0000561949a5f448[8byte]. When I put in those bytes in gdb, it gave me:
(gdb) x/10xb a
0x7fffffffd9c1: 0x48 0xa1 0x48 0xf4 0xa5 0x49 0x19 0x56
0x7fffffffd9c9: 0x00 0x00
(gdb) x/10i a
0x7fffffffd9c1: movabs 0x561949a5f448,%rax
0x7fffffffd9cb: add %al,(%rax)
(gdb) x/10xb a
0x7fffffffd9c1: 0x48 0x8b 0x05 0x05 0x3d 0x69 0x40 0x00
0x7fffffffd9c9: 0x00 0x00
(gdb) x/10i a
0x7fffffffd9c1: mov 0x40693d05(%rip),%rax # 0x8000406916cd
0x7fffffffd9c8: add %al,(%rax)
I don't know whether this issue is limited to only gencode. I don't see any non-gencode tests that trigger state restore with L0_filter_until_instrs also set.
Disassembling in isolation with drdecode does not reproduce what you describe. DR decodes the two as expected with the first absolute and the 2nd pc-rel:
$ disasm 48 a1 48 f4 a5 49 19 56 00 00
llvm-mc: 0x48 0xa1 0x48 0xf4 0xa5 0x49 0x19 0x56 0x00 0x00 movabsq 94666609783880, %rax
capstone: a148 f448 movabs rax, qword ptr [0x561949a5f448]
bfd: 48 a1 48 f4 a5 49 19 movabs 0x561949a5f448,%rax
xed: 0x48 0xa1 0x48 0xf4 0xa5 0x49 0x19 0x56 0x00 0x00 mov rax, qword ptr [0x561949a5f448]
DynamoRIO: 48 a1 48 f4 a5 49 19 mov rax, qword ptr [0x0000561949a5f448] 56 00 00
$ disasm 48 8b 05 05 d0 90 40
llvm-mc: 0x48 0x8b 0x05 0x05 0xd0 0x90 0x40 movq 1083232261(%rip), %rax
capstone: 8b48 0505 mov rax, qword ptr [rip + 0x4090d005]
bfd:
xed: 0x48 0x8b 0x05 0x05 0xd0 0x90 0x40 mov rax, qword ptr [rip+0x4090d005]
DynamoRIO: 48 8b 05 05 d0 90 40 mov rax, <rel> qword ptr [0x000056317918555c]
It does look like only the first mov instr is mismatching between the two versions:
Provided ilist:
ilist for recreation:
TAG 0x00007f1da7115000
+0 m4 @0x00007f1b66bf4af0 48 a1 48 54 14 9b c5 mov <rel> 0x000055c59b145448[8byte] -> %rax
55 00 00
+10 m4 @0x00007f1b66bf0a18 <label note=0x0000000000000008>
+10 m4 @0x00007f1b66bf3ff8 48 3d 03 00 00 00 cmp %rax $0x0000000000000003
+16 m4 @0x00007f1b66bf13c8 0f 85 fa ff ff ff jnz @0x00007f1b66bf3998[8byte]
+22 L3 @0x00007f1b66bf0b28 90 nop
+23 L3 @0x00007f1b66bf4818 b8 42 07 00 00 mov $0x00000742 -> %eax
+28 L3 @0x00007f1b66bf4570 b8 27 04 00 00 mov $0x00000427 -> %eax
...
Decoded manually from faulting fragment:
cpc=55c55aab173c, target_cache=55c55aab1869, mov <rel> 0x000055c59b145448[8byte] -> %rax bytes= 48 8b 05 05 3d 69 40
cpc=55c55aab1743, target_cache=55c55aab1869, cmp %rax $0x0000000000000003 bytes= 48 3d 03 00 00 00
cpc=55c55aab1749, target_cache=55c55aab1869, jnz $0x000055c55aab176b bytes= 0f 85 1c 00 00 00
cpc=55c55aab174f, target_cache=55c55aab1869, nop bytes= 90
cpc=55c55aab1750, target_cache=55c55aab1869, mov $0x00000742 -> %eax bytes= b8 42 07 00 00
cpc=55c55aab1755, target_cache=55c55aab1869, mov $0x00000427 -> %eax bytes= b8 27 04 00 00
...
That mov is likely the one added by drbbdup. Not sure if drbbdup is adding a different kind of mov during the translation event or if DR is supposed to mangle that longer 10-byte mov in some later phase: both cases would point to a bug I think.
Looking at prior logs from when the fragment was created, it does look like the mov is modified into the 7-byte version before the fragment is added to the bb cache. Note how after mangling, it's still the 10-byte version.
exit_branch_type=0x0 bb->exit_target=0x00007f1da7115017
fragment writes OF prior to reading it!
bb ilist after mangling:
TAG 0x00007f1da7115000
+0 m4 @0x00007f1b66c3df60 48 a1 48 54 14 9b c5 mov <rel> 0x000055c59b145448[8byte] -> %rax // Still the 10-byte version
55 00 00
+10 m4 @0x00007f1b66c3e058 <label note=0x0000000000000008>
+10 m4 @0x00007f1b66c3d618 48 3d 03 00 00 00 cmp %rax $0x0000000000000003
+16 m4 @0x00007f1b66c3ea88 0f 85 fa ff ff ff jnz @0x00007f1b66c3e5e8[8byte]
+22 L3 @0x00007f1b66c3e4d8 90 nop
...
END 0x00007f1da7115000
fcache_add_fragment to Basic block (shared) cache (size 8KB): F6 w/ size 800 (=> 808)
find_free_list_slot: 808 bytes
added F6 to unfilled unit @0x000055c55aab173c (1444 [/57344] bytes left now)
nop_pad_ilist: F6 @0x000055c55aab1a54 cti shift needed: 0
exit_branch_type=0x0 target=0x00007f1da7115017 l->flags=0x9801
vm_area_add_fragment for F6(0x00007f1da7115000)
linking new fragment F6(0x00007f1da7115000)
linking incoming links for F6(0x00007f1da7115000)
linking outgoing links for F6(0x00007f1da7115000)
Created future fragment 0x00007f1da7115017 w/ flags 0x01000101
hashtable_fragment_add(0x00007f1da7115017) mask=0x00000000000003ff offset=0 trying 0x000000000000000d:
hashtable_fragment_add: added 0x00007f1da7115017 to shared_future at table[13]
future-linking F6(0x00007f1da7115000).0x000055c55aab1a51 -> (0x00007f1da7115017)
hashtable_fragment_add(0x00007f1da7115000) mask=0x00000000000003ff offset=0 trying 0x0000000000000331:
hashtable_fragment_add: added 0x00007f1da7115000 to shared_bb at table[817]
Fragment 6, tag 0x00007f1da7115000, flags 0x1000630, shared, size 794:
0x000055c55aab173c 48 8b 05 05 3d 69 40 mov <rel> 0x000055c59b145448[8byte] -> %rax // 7 byte now
0x000055c55aab1743 48 3d 03 00 00 00 cmp %rax $0x0000000000000003
0x000055c55aab1749 0f 85 1c 00 00 00 jnz $0x000055c55aab176b
0x000055c55aab174f 90 nop
0x000055c55aab1750 b8 42 07 00 00 mov $0x00000742 -> %eax
``