dynamorio icon indicating copy to clipboard operation
dynamorio copied to clipboard

TZCNT is incorrectly decoded

Open dvyukov opened this issue 3 years ago • 2 comments

Decoding this byte sequence: f3 f2 0f bc f2 DynamoRIO: tzcnt %edx -> %esi Capstone: bsf esi, edx llvm-mc: invalid instruction encoding objdump:

   0:	f3 f2 0f bc          	repz (bad) 
   4:	f2                   	repnz

Intel manual says TZCNT encodings are:

  F3 0F BC /r
  F3 REX.W 0F BC /r

Executing this instruction with different values of RDX always yields 0 in RSI, so effectively it seems to work as some kind of non-trapping nop (?).

If it's an invalid instruction for my use case it would be more useful if DynamoRIO would reject to decode it (or decode to something that would agree with CPU behavior, e.g. OP_nop).

On commit 6e88738383e78c0be9c3dda373e43f311425b862.

dvyukov avatar Apr 05 '22 09:04 dvyukov

Adding xed:

xed:       0xf3 0xf2 0x0f 0xbc 0xf2   bsf esi, edx

Executing this instruction with different values of RDX always yields 0 in RSI, so effectively it seems to work as some kind of non-trapping nop (?).

If it writes a 0 into RSI that does not sound like a NOP?

derekbruening avatar Apr 05 '22 15:04 derekbruening

Doing more tests: Initially RSI is always 0xabababababababab When RDX = 0, RSI = 0xabababababababab, ZF is set When RDX = 1, RSI = 0, ZF is not set When RDX = 0x10, RSI = 4 When RDX = 0x80, RSI = 7 When RDX = 0x80000000, RSI = 0x1f

So it looks like XED is right and this is BSF. BSF encodings are "0F BC /r" and "REX.W + 0F BC /r". So it's BSF with some pointless prefixes?

dvyukov avatar Apr 06 '22 08:04 dvyukov