Ignore Host Compiler

[jpc0016]

Good afternoon,

While attempting to build McSema on Ubuntu 19.04, the #binary-lifting community uncovered an issue with runtimes ignoring use-host-compiler and defaulting to prebuilt ones.

Steps taken:

I modified function GetUbuntuOsVersion in build.sh to look for 'disco' version by adding:

;
   disco)
     USE_HOST_COMPILER=1
     OS_VERSION=ubuntu1804
     return 0

I received this fatal error.

[ 75%] Building 32-bit runtime
~/mcsema-ve/remill/tools/mcsema/mcsema/Arch/X86/Runtime/Runtime.cpp:17:10: fatal error: 'cstdio' file not found
#include <cstdio>
        ^~~~~~~~
1 error generated.

So Runtime.cpp cannot find cstudio for some reason. As Aiethel described, I added a size check to the runtime block in CMakeLists.txt (around Line 211) like so:

# Create a runtime.
if(CMAKE_SIZEOF_VOID_P EQUAL 4)
  add_custom_command(
    OUTPUT runtime_32.o
    COMMAND "${CMAKE_BC_COMPILER}" -std=gnu++11 -isystem "${REMILL_SOURCE_DIR}" -isystem "${MCSEMA_SOURCE_DIR}" -m32 -fPIC -c "${CMAKE_CURRENT_SOURCE_DIR}/Runtime.cpp" -o runtime_32.o
    DEPENDS Runtime.cpp
    COMMENT "Building 32-bit runtime"
  )
endif()

I still get 95% build progress, but cannot help feeling there is something missing. I'm not sure why cannot be found when I ran it a second ago in an example program. I appreciate the help everyone!

[pgoodman]

I think the crux of this issue is that our compiled clang doesn't discover all the same default include paths as the host compiler, or puts the ordering of them differently.

Ideally, we'd run something like: echo | ${CMAKE_CXX_COMMAND} -E -Wp,-v -, then parse out the include paths from there, pass them into the above COMMAND, and then after include -nostdinc and -nostdinc++.

[jpc0016]

So ideally -nostdinc and -nostdinc++ need to be in the COMMAND like so?

COMMAND "${CMAKE_BC_COMPILER}" -std=gnu++11 -isystem "${REMILL_SOURCE_DIR}" -isystem "${MCSEMA_SOURCE_DIR}" -m32 -fPIC -c "${CMAKE_CURRENT_SOURCE_DIR}/Runtime.cpp" -o runtime_32.o -nostdinc -nostdinc++

[pgoodman]

It's more than that. Here's an example:

pag$ echo | `which cc` -E -Wp,-v -
clang -cc1 version 10.0.1 (clang-1001.0.46.4) default target x86_64-apple-darwin18.7.0
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/10.0.1/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
 /usr/include
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
End of search list.
# 1 "<stdin>"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 361 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "<stdin>" 2

So we would need -isystem each of: /usr/local/include /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/10.0.1/include /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include /usr/include /usr/include

Framework directories could be added but shouldn't really be needed.

[pgoodman]

Really what we need is, instead of invoking COMMAND "${CMAKE_BC_COMPILER}" - ..., we should invoke a Python script that will do the right thing.