Problem with x86 CDQE-family instructions

[jbaramidze]

DynamoRIO interprets source and destination registers for CDQE-family instructions (cbw, cwde, cdqe) incorrectly. For example, for cdqe, source register is EAX while DR prints AX. Also, for cbw, source is AL while DR prints AX.

[derekbruening]

Apparently I made some private notes when this was filed but never posted them. Pasting here now:

The manual says:

98         cbw   ax <- al             OP_cwde
98         cwde  eax <- ax            OP_cwde
rex.w+98   cdqe  rax <- eax           OP_cwde
99         cwd   dx:ax <- ax          OP_cdq
99         cdq   edx:eax <- eax       OP_cdq
rex.w+99   cqo   rdx:rax <- rax       OP_cdq

DR says from allasm_x64.s:

        cbw
        cwde
        cdqe
        cwd
        cdq
        cqo
=>
  0x0000000000400150  66 98                data16 cwde   %ax -> %ax
  0x0000000000400152  98                   cwde   %ax -> %eax
  0x0000000000400153  48 98                cwde   %ax -> %rax
  0x0000000000400155  66 99                data16 cdq    %ax -> %dx
  0x0000000000400157  99                   cdq    %eax -> %edx
  0x0000000000400158  48 99                cdq    %rax -> %rdx

It's the hardcoded "ax" here:

    {OP_cwde, 0x980000, "cwde", eAX, xx, ax, xx, xx, no, x, END_LIST},/*16-bit=="cbw", src is al not ax; FIXME: newer gdb calls it "cwtl"?!?*/

Needs to shrink to "al" and expand to "eax". I guess mark as TYPE_VAR_REG and have resolve_var_reg() check whether it's a 16-bit reg?

[derekbruening]

The data prefix also produces the wrong thing, from #5448:

66 98	data16 cwde	%ax -> %ax 

Should be:

66 98	data16 cwde	%al -> %ax 

[khuey]

I think we would need to add OPSZ_2_rex4_short1 and an equivalent TYPE_VAR_REG_SOMETHING?

[derekbruening]

I think we would need to add OPSZ_2_rex4_short1 and an equivalent TYPE_VAR_REG_SOMETHING?

IIRC in the table for a hardcoded register (eax/rax here) there is no size and that field instead holds the register, so there may be nowhere to use a new size class. But yes it would operate as though it were that 2_rex4_short1.