reko icon indicating copy to clipboard operation
reko copied to clipboard

System calls need a reworking

Open uxmal opened this issue 4 years ago • 0 comments

The way Reko handles system calls is inconsistent across platforms, and is causing problems resolving #918. Not only that, Reko currently cannot handle the situation on x86 Linux where there are multiple ways to invoke a service.

To resolve this I propose the following:

  • Introduce a new property on IntrinsicProcedure called IsSystemCall
  • When encountering a system call instruction, each architecture generates a SideEffect invoking such an IsSystemCall intrinsic procedure whose name can be architecture specific, like __int, __sysenter or __syscall for x86, __trap for PPC, __sc, etc.
  • The [[reko::service]] attribute needs to be extended to support specification of the name of the intrinsic procedure:
[[reko::service(inst="__int" vector=0x80, regs={eax:1})]]
[[reko::service(inst="__syscall" regs={eax:1})]] 
    [[reko::convention(x86kernel)]] long sys_exit(int error_code);
  • The corresponding XML syntax needs adjusting too. Every <service> element needs to be able to indicate support for multiple instructions for the same service:
<service name="sys_exit">
    <instr name="__int">
       <vector>80</vector>
       <regvalue reg="eax">1</regvalue>
    </instr>
    <instr name="__syscall">
       <regvalue reg="eax">1</regvalue>
    </instr>
    <signature>
      <return>eax</return>
      <prim domain="SignedInt" size="4"/>
      <arg name="fd">
        <reg>ebx</reg>
        <ptr><prim domain="Character" size="4"/></ptr>
      </arg>
    </signature>
</service>

With these changes implemented, the scanner can detect a system call, use the instruction name to look up the appropriate service, and inject the proper signature.

uxmal avatar Dec 11 '21 20:12 uxmal