angr Windows inet_ntoa() calling-convention exception

Describe the bug. angr crashes when it attempts to simulate ws2_32.dll's inet_ntoa(), with TypeError: I don't know how to lay out a union None. It seems that the code for dealing with calling conventions can't handle this function's auto-generated definition, but I'm not familiar enough with the code to understand the root cause.

Environment Information. I reinstalled angr in a new virtual environment to check that this happens on the latest version:

$ python3 -m angr.misc.bug_report
[snip]/.venv_python3/lib/python3.8/site-packages/angr/misc/bug_report.py:1: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
angr environment report
=============================
Date: 2022-08-24 18:00:07.034831
Running in virtual environment at [snip]/.venv_python3
Platform: linux-x86_64
Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) 
[GCC 9.4.0]
######## angr #########
Python found it in [snip]/.venv_python3/lib/python3.8/site-packages/angr
Pip version angr 9.2.15
######## ailment #########
Python found it in [snip]/.venv_python3/lib/python3.8/site-packages/ailment
Pip version ailment 9.2.15
######## cle #########
Python found it in [snip]/.venv_python3/lib/python3.8/site-packages/cle
Pip version cle 9.2.15
######## pyvex #########
Python found it in [snip]/.venv_python3/lib/python3.8/site-packages/pyvex
Pip version pyvex 9.2.15
######## claripy #########
Python found it in [snip]/.venv_python3/lib/python3.8/site-packages/claripy
Pip version claripy 9.2.15
######## archinfo #########
Python found it in [snip]/.venv_python3/lib/python3.8/site-packages/archinfo
Pip version archinfo 9.2.15
######## z3 #########
Python found it in [snip]/.venv_python3/lib/python3.8/site-packages/z3
Pip version z3-solver 4.10.2.0
######## unicorn #########
Python found it in [snip]/.venv_python3/lib/python3.8/site-packages/unicorn
Pip version unicorn 1.0.2rc4
######### Native Module Info ##########
angr: <CDLL '[snip]/.venv_python3/lib/python3.8/site-packages/angr/state_plugins/../lib/angr_native.so', handle 1d64460 at 0x7ff54f9eb220>
unicorn: <CDLL '[snip]/.venv_python3/lib/python3.8/site-packages/unicorn/lib/libunicorn.so', handle 16af8c0 at 0x7ff54ed0fa30>
pyvex: <cffi.api._make_ffi_library.<locals>.FFILibrary object at 0x7ff54f931df0>
z3: NOT FOUND
$

To Reproduce. Example binary compiled with VS 2022 in 32-bit release mode. Source code:

#define _WINSOCK_DEPRECATED_NO_WARNINGS

#include <winsock2.h>
#include <stdio.h>

#pragma comment(lib, "ws2_32")

int main()
{
    struct in_addr addr;
    addr.S_un.S_addr = 0x0100007f;
    printf("Localhost is %s", inet_ntoa(addr));  // prints "Localhost is 127.0.0.1"
}

Python script:

import angr
proj = angr.Project('TestApp.exe', auto_load_libs=False)
state = proj.factory.blank_state()
sim_mgr = proj.factory.simgr(state)
sim_mgr.run()

Additional context. Output from the Python script:

$ python3 inet_ntoa_test.py
WARNING | 2022-08-24 18:03:07,886 | angr.storage.memory_mixins.default_filler_mixin | The program is accessing register with an unspecified value. This could indicate unwanted behavior.
WARNING | 2022-08-24 18:03:07,886 | angr.storage.memory_mixins.default_filler_mixin | angr will cope with this by generating an unconstrained symbolic variable and continuing. You can resolve this by:
WARNING | 2022-08-24 18:03:07,886 | angr.storage.memory_mixins.default_filler_mixin | 1) setting a value to the initial state
WARNING | 2022-08-24 18:03:07,886 | angr.storage.memory_mixins.default_filler_mixin | 2) adding the state option ZERO_FILL_UNCONSTRAINED_{MEMORY,REGISTERS}, to make unknown regions hold null
WARNING | 2022-08-24 18:03:07,886 | angr.storage.memory_mixins.default_filler_mixin | 3) adding the state option SYMBOL_FILL_UNCONSTRAINED_{MEMORY,REGISTERS}, to suppress these messages.
WARNING | 2022-08-24 18:03:07,886 | angr.storage.memory_mixins.default_filler_mixin | Filling register esi with 4 unconstrained bytes referenced from 0x401681 (offset 0x1681 in TestApp.exe (0x401681))
WARNING | 2022-08-24 18:03:07,887 | angr.storage.memory_mixins.default_filler_mixin | Filling register edi with 4 unconstrained bytes referenced from 0x401682 (offset 0x1682 in TestApp.exe (0x401682))
WARNING | 2022-08-24 18:03:07,898 | angr.storage.memory_mixins.default_filler_mixin | Filling register ebp with 4 unconstrained bytes referenced from 0x401980 (offset 0x1980 in TestApp.exe (0x401980))
WARNING | 2022-08-24 18:03:07,899 | angr.storage.memory_mixins.default_filler_mixin | Filling register ebx with 4 unconstrained bytes referenced from 0x40198a (offset 0x198a in TestApp.exe (0x40198a))
WARNING | 2022-08-24 18:03:07,994 | angr.storage.memory_mixins.default_filler_mixin | Filling memory at 0x7fff0000 with 4 unconstrained bytes referenced from 0x40129a (offset 0x129a in TestApp.exe (0x40129a))
WARNING | 2022-08-24 18:03:08,099 | angr.engines.successors | Exit state has over 256 possible solutions. Likely unconstrained; skipping. <BV32 mem_7fff0000_5_32{UNINITIALIZED}>
WARNING | 2022-08-24 18:03:08,103 | angr.storage.memory_mixins.default_filler_mixin | Filling memory at 0x7ffeffd9 with 3 unconstrained bytes referenced from 0x4011be (offset 0x11be in TestApp.exe (0x4011be))
WARNING | 2022-08-24 18:03:08,200 | angr.storage.memory_mixins.default_filler_mixin | Filling memory at 0xffffffff with 1 unconstrained bytes referenced from 0x401219 (offset 0x1219 in TestApp.exe (0x401219))
WARNING | 2022-08-24 18:03:08,200 | angr.storage.memory_mixins.default_filler_mixin | Filling memory at 0x0 with 3 unconstrained bytes referenced from 0x401219 (offset 0x1219 in TestApp.exe (0x401219))
WARNING | 2022-08-24 18:03:08,277 | angr.storage.memory_mixins.default_filler_mixin | Filling memory at 0xffff8000 with 4 unconstrained bytes referenced from 0x401222 (offset 0x1222 in TestApp.exe (0x401222))
Traceback (most recent call last):
  File "inet_ntoa_test.py", line 7, in <module>
    sim_mgr.run()
  File "[snip]/.venv_python3/lib/python3.8/site-packages/angr/sim_manager.py", line 318, in run
    self.step(stash=stash, **kwargs)
  File "[snip]/.venv_python3/lib/python3.8/site-packages/angr/misc/hookset.py", line 90, in __call__
    result = current_hook(self.func.__self__, *args, **kwargs)
  File "[snip]/.venv_python3/lib/python3.8/site-packages/angr/exploration_techniques/suggestions.py", line 41, in step
    simgr = simgr.step(stash=stash, **kwargs)
  File "[snip]/.venv_python3/lib/python3.8/site-packages/angr/misc/hookset.py", line 95, in __call__
    return self.func(*args, **kwargs)
  File "[snip]/.venv_python3/lib/python3.8/site-packages/angr/sim_manager.py", line 407, in step
    successors = self.step_state(state, successor_func=successor_func, error_list=error_list, **run_args)
  File "[snip]/.venv_python3/lib/python3.8/site-packages/angr/sim_manager.py", line 448, in step_state
    successors = self.successors(state, successor_func=successor_func, **run_args)
  File "[snip]/.venv_python3/lib/python3.8/site-packages/angr/sim_manager.py", line 496, in successors
    return self._project.factory.successors(state, **run_args)
  File "[snip]/.venv_python3/lib/python3.8/site-packages/angr/factory.py", line 60, in successors
    return self.default_engine.process(*args, **kwargs)
  File "[snip]/.venv_python3/lib/python3.8/site-packages/angr/engines/vex/light/slicing.py", line 19, in process
    return super().process(*args, **kwargs)
  File "[snip]/.venv_python3/lib/python3.8/site-packages/angr/engines/engine.py", line 160, in process
    self.process_successors(self.successors, **kwargs)
  File "[snip]/.venv_python3/lib/python3.8/site-packages/angr/engines/failure.py", line 21, in process_successors
    return super().process_successors(successors, **kwargs)
  File "[snip]/.venv_python3/lib/python3.8/site-packages/angr/engines/syscall.py", line 18, in process_successors
    return super().process_successors(successors, **kwargs)
  File "[snip]/.venv_python3/lib/python3.8/site-packages/angr/engines/hook.py", line 61, in process_successors
    return self.process_procedure(state, successors, procedure, **kwargs)
  File "[snip]/.venv_python3/lib/python3.8/site-packages/angr/engines/procedure.py", line 37, in process_procedure
    inst = procedure.execute(state, successors, ret_to=ret_to, arguments=arguments)
  File "[snip]/.venv_python3/lib/python3.8/site-packages/angr/sim_procedure.py", line 242, in execute
    sim_args = [inst.cc.next_arg(inst.arg_session, ty).get_value(inst.state) for ty in inst.prototype.args]
  File "[snip]/.venv_python3/lib/python3.8/site-packages/angr/sim_procedure.py", line 242, in <listcomp>
    sim_args = [inst.cc.next_arg(inst.arg_session, ty).get_value(inst.state) for ty in inst.prototype.args]
  File "[snip]/.venv_python3/lib/python3.8/site-packages/angr/calling_conventions.py", line 1132, in next_arg
    return refine_locs_with_struct_type(self.arch, locs, arg_type)
  File "[snip]/.venv_python3/lib/python3.8/site-packages/angr/calling_conventions.py", line 133, in refine_locs_with_struct_type
    locs = {
  File "[snip]/.venv_python3/lib/python3.8/site-packages/angr/calling_conventions.py", line 134, in <dictcomp>
    field: refine_locs_with_struct_type(
  File "[snip]/.venv_python3/lib/python3.8/site-packages/angr/calling_conventions.py", line 139, in refine_locs_with_struct_type
    raise TypeError("I don't know how to lay out a %s" % arg_type)
TypeError: I don't know how to lay out a union None

Aug 24 '22 22:08 stevenskevin

While this is a bug and we would like to get to it at some point, it is low-priority as we currently have a lot on our plate.

Aug 31 '22 16:08 ltfish

I encountered this issue while running CFGEmulated's over Windows binaries en masse, which raised a terminating at Exception all instances of ws2_32's inet_ntoa with the same error/Traceback as the OP issue

To help characterize the scope of this particular issue, as well as aid reproducibility I created the following test snippet.

At its core, it simply loads all win32 SimProcedure definitions in an x86 architecture setup with blank state, then attempts to .execute() each with default (None) parameters, capturing the TypeErrors related to SimUnions in function arguments

from multiprocessing import Pool
import angr
from angr.procedures import SIM_PROCEDURES, SIM_LIBRARIES
import logging; logging.disable(logging.CRITICAL);

#angr.procedures.definitions.load_all_definitions()
angr.procedures.definitions.load_win32api_definitions()

x86 = angr.archinfo.arch_x86.ArchX86()
state = angr.sim_state.SimState(arch=x86)

def search_lib(LIB):
    proc_names = SIM_LIBRARIES[LIB].prototypes.keys()
    for name in proc_names:
        try:
            ws_proc = SIM_LIBRARIES[LIB].get(name,x86)
            ws_proc.execute(state)
        except TypeError as e:
            if "how to lay out a union None" in repr(e):
                print("TypeError!",LIB,name, type(e), e)
        except Exception:
            pass

if __name__ == "__main__":
    P = Pool()
    P.map(search_lib, list(SIM_LIBRARIES.keys()))

Results show a number of other uncommon libraries contain SimUnions passed into their parameters, with the only other semi-substantial occurrence being kernel32's "SetFilePointerEx" with its LARGE_INTEGER union

TypeError! winusb.dll WinUsb_GetAdjustedFrameNumber <class 'TypeError'> I don't know how to lay out a union None
TypeError! cldapi.dll CfHydratePlaceholder <class 'TypeError'> I don't know how to lay out a union None
TypeError! cldapi.dll CfDehydratePlaceholder <class 'TypeError'> I don't know how to lay out a union None
TypeError! rtm.dll RtmConvertIpv6AddressAndLengthToNetAddress <class 'TypeError'> I don't know how to lay out a union None
TypeError! cldapi.dll CfGetPlaceholderRangeInfo <class 'TypeError'> I don't know how to lay out a union None
TypeError! cldapi.dll CfReportProviderProgress <class 'TypeError'> I don't know how to lay out a union None
TypeError! cldapi.dll CfReportProviderProgress2 <class 'TypeError'> I don't know how to lay out a union None
TypeError! activeds.dll SecurityDescriptorToBinarySD <class 'TypeError'> I don't know how to lay out a union None
TypeError! wofutil.dll WofWimEnumFiles <class 'TypeError'> I don't know how to lay out a union None
TypeError! wofutil.dll WofWimSuspendEntry <class 'TypeError'> I don't know how to lay out a union None
TypeError! wofutil.dll WofWimRemoveEntry <class 'TypeError'> I don't know how to lay out a union None
TypeError! wofutil.dll WofWimUpdateEntry <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarUI1FromCy <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarI2FromCy <class 'TypeError'> I don't know how to lay out a union None
TypeError! winbio.dll WinBioRemoveCredential <class 'TypeError'> I don't know how to lay out a union None
TypeError! winbio.dll WinBioGetCredentialState <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarI4FromCy <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarI8FromCy <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarR4FromCy <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarR8FromCy <class 'TypeError'> I don't know how to lay out a union None
TypeError! uiautomationcore.dll TextRange_FindAttribute <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarDateFromCy <class 'TypeError'> I don't know how to lay out a union None
TypeError! uiautomationcore.dll ItemContainerPattern_FindItemByProperty <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarBstrFromCy <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarBoolFromCy <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarI1FromCy <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarUI2FromCy <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarUI4FromCy <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarUI8FromCy <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarDecFromCy <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarCyAdd <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarCyMul <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarCyMulI4 <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarCyMulI8 <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarCySub <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarCyAbs <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarCyFix <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarCyInt <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarCyNeg <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarCyRound <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarCyCmp <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll VarCyCmpR8 <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll OleLoadPictureFile <class 'TypeError'> I don't know how to lay out a union None
TypeError! oleaut32.dll OleLoadPictureFileEx <class 'TypeError'> I don't know how to lay out a union None
TypeError! ole32.dll CoRevokeInitializeSpy <class 'TypeError'> I don't know how to lay out a union None
TypeError! ws2_32.dll inet_ntoa <class 'TypeError'> I don't know how to lay out a union None
TypeError! kernel32.dll SetFilePointerEx <class 'TypeError'> I don't know how to lay out a union None

The common prototype characteristic between each of these functions is that an argument directly contains a SimUnion type instead of a pointer->SimUnion

Aug 05 '23 04:08 AlLongley

The issue as I understand it is that angr.calling_conventions.refine_locs_with_struct_type has no handler for SimUnion types, most likely as they are hard to characterize as they exist as a placeholder which could contain any of the listed types within.

As it is crashing my scanning runs and I'm less interested in the most elegant solution for resolving this very specific edge case, I intend to band-aid over it by simply treating a SimUnion in this context as being equal to the first instance of its longest constituent member

    if isinstance(arg_type, SimUnion):
        for member in arg_type.members.items():
            if member[1].size == arg_type.size:
                break
        return refine_locs_with_struct_type(arch, locs, member[1], offset)

Aug 05 '23 05:08 AlLongley

Is this issue resolved after that PR is merged?

Aug 08 '23 18:08 rhelmot

Fixed for me against the above provided example and independently on other WS2_32 based windows binaries of mine

Aug 09 '23 00:08 AlLongley

angr angr copied to clipboard

Windows inet_ntoa() calling-convention exception

angr
angr copied to clipboard