mono icon indicating copy to clipboard operation
mono copied to clipboard

Mono fails to build as cross compiler

Open ryan-bunker opened this issue 4 years ago • 8 comments

It appears that building mono from source as a cross compiler is broken. I've followed the instructions for building on macOs (https://www.mono-project.com/docs/compiling-mono/mac/). When using this configuration line:

./configure --prefix=$HOME/opt/cross --disable-nls --target=i686-linux-elf

I get the following errors when running make:

abi.c:33:1: error: use of empty enum
};
^
abi.c:36:2: error: use of undeclared identifier 'MONO_ALIGN_value_gint8'; did you mean 'MONO_ALIGN_gint8'?
        MONO_ALIGN_value_gint8,
        ^~~~~~~~~~~~~~~~~~~~~~
        MONO_ALIGN_gint8
abi.c:14:4: note: expanded from macro 'DECLARE_ABI_DETAILS'
        { I8, I16, I32, I64, F32, F64, PTR }    \
          ^
../../mono/metadata/abi-details.h:27:2: note: 'MONO_ALIGN_gint8' declared here
        MONO_ALIGN_gint8,
        ^
abi.c:37:2: error: use of undeclared identifier 'MONO_ALIGN_value_gint16'; did you mean 'MONO_ALIGN_gint16'?
        MONO_ALIGN_value_gint16,
        ^~~~~~~~~~~~~~~~~~~~~~~
        MONO_ALIGN_gint16
abi.c:14:8: note: expanded from macro 'DECLARE_ABI_DETAILS'
        { I8, I16, I32, I64, F32, F64, PTR }    \
              ^
../../mono/metadata/abi-details.h:28:2: note: 'MONO_ALIGN_gint16' declared here
        MONO_ALIGN_gint16,
        ^
abi.c:38:2: error: use of undeclared identifier 'MONO_ALIGN_value_gint32'; did you mean 'MONO_ALIGN_gint32'?
        MONO_ALIGN_value_gint32,
        ^~~~~~~~~~~~~~~~~~~~~~~
        MONO_ALIGN_gint32
abi.c:14:13: note: expanded from macro 'DECLARE_ABI_DETAILS'
        { I8, I16, I32, I64, F32, F64, PTR }    \
                   ^
../../mono/metadata/abi-details.h:29:2: note: 'MONO_ALIGN_gint32' declared here
        MONO_ALIGN_gint32,
        ^
abi.c:39:2: error: use of undeclared identifier 'MONO_ALIGN_value_gint64'; did you mean 'MONO_ALIGN_gint64'?
        MONO_ALIGN_value_gint64,
        ^~~~~~~~~~~~~~~~~~~~~~~
        MONO_ALIGN_gint64
abi.c:14:18: note: expanded from macro 'DECLARE_ABI_DETAILS'
        { I8, I16, I32, I64, F32, F64, PTR }    \
                        ^
../../mono/metadata/abi-details.h:30:2: note: 'MONO_ALIGN_gint64' declared here
        MONO_ALIGN_gint64,
        ^
abi.c:40:2: error: use of undeclared identifier 'MONO_ALIGN_value_float'; did you mean 'MONO_ALIGN_float'?
        MONO_ALIGN_value_float,
        ^~~~~~~~~~~~~~~~~~~~~~
        MONO_ALIGN_float
abi.c:14:23: note: expanded from macro 'DECLARE_ABI_DETAILS'
        { I8, I16, I32, I64, F32, F64, PTR }    \
                             ^
../../mono/metadata/abi-details.h:31:2: note: 'MONO_ALIGN_float' declared here
        MONO_ALIGN_float,
        ^
abi.c:41:2: error: use of undeclared identifier 'MONO_ALIGN_value_double'; did you mean 'MONO_ALIGN_double'?
        MONO_ALIGN_value_double,
        ^~~~~~~~~~~~~~~~~~~~~~~
        MONO_ALIGN_double
abi.c:14:28: note: expanded from macro 'DECLARE_ABI_DETAILS'
        { I8, I16, I32, I64, F32, F64, PTR }    \
                                  ^
../../mono/metadata/abi-details.h:32:2: note: 'MONO_ALIGN_double' declared here
        MONO_ALIGN_double,
        ^
abi.c:42:2: error: use of undeclared identifier 'MONO_ALIGN_value_gpointer'; did you mean 'MONO_ALIGN_gpointer'?
        MONO_ALIGN_value_gpointer)
        ^~~~~~~~~~~~~~~~~~~~~~~~~
        MONO_ALIGN_gpointer
abi.c:14:33: note: expanded from macro 'DECLARE_ABI_DETAILS'
        { I8, I16, I32, I64, F32, F64, PTR }    \
                                       ^
../../mono/metadata/abi-details.h:33:2: note: 'MONO_ALIGN_gpointer' declared here
        MONO_ALIGN_gpointer,
        ^
8 errors generated.

I'm able to compile without issue if I leave off the --target flag. Looking at the source code for abi.c it appears that when MONO_CROSS_COMPILE is defined along with TARGET_X86, compiler errors are introduced.

I'm building on MacOS 10.15.5 using the tarball source for version 6.10.0.105. I've also tried version 6.8.0.123, as well as doing the build on Windows WSL (Ubuntu 20.04).

ryan-bunker avatar Jun 27 '20 16:06 ryan-bunker

Hi @ryan-bunker, thanks for the bug report.

Could you tell me what you're trying to accomplish? Is the goal to build a working Mono runtime on a Mac that will run on 32-bit Linux?

In that case you should specify --build=x86_64-apple-darwin19.6.0 and --host=i686-linux-elf (note: host, not target. target is genrally only meaningful when you want to build Mono'S Ahead of Time compiler to run on one machine while producing AOT images for another). Note that you will need to have gcc or clang and binutils (ld, as, etc) that target linux available. You may also need to specify CC, CXX, LD, AS etc flags to configure if it can't find the right utils by itself.

If you are building binaries for 64-bit linux (this is more common nowadays) you want --host=amd64-linux-elf.

lambdageek avatar Oct 22 '20 14:10 lambdageek

Hello,

I am facing the same problem, the error message is exactly the same as above. What I am trying to achieve is to compile mono as AOT cross-compiler (I want to precompile ARM binaries on x86_64 build host).

Mono version: 6.8.0.123 Configure args: --build=x86_64-linux --host=x86_64-linux --target=arm-poky-linux-gnueabi --disable-silent-rules --disable-dependency-tracking mono_cv_uscore=no --with-sigaltstack=no --with-mcs-docs=no --disable-nls

I am compiling with Yocto build system on Linux.

klasyc avatar Aug 08 '22 15:08 klasyc

@klasyc in order to cross-compile you need to generate an "offsets header file" using offsets-tool and pass the path of the offsets file to configure using --with-cross-offsets=xyz.h

For example this is how the mono builds do it when creating cross-compilers https://github.com/mono/mono/blob/ef848cfa83ea16b8afbd5b933968b1838df19505/sdks/builds/runtime.mk#L225

you will need to pass at least --abi=, --libclang= and -sysroot= arguments to offsets-tool.

lambdageek avatar Aug 08 '22 16:08 lambdageek

Thank you for suggestion, @lambdageek! Now I am trying to use the offsets-tool, but it fails and I cannot find any documentation. That's pity because I am not sure what exactly this tool is supposed to do. I assume it should provide some kind of header file which is generated from the libraries I have already compiled. My current output is:

ubuntu@docker-desktop:~/yocto/yocto-image-eda/build/tmp/work/x86_64-linux/mono-cross-armv7ahf-neon/6.8.0.123-r0/mono-6.8.0.123$ python3 mono/tools/offsets-tool/offsets-tool.py --abi=arm-linux-gnueabihf --outfile=offsets.h --monodir=. --targetdir=. --libclang=/usr/lib/x86_64-linux-gnu/libclang-8.so.1 --sysroot=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/beaglebone-lcd/ --prefix=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/lib/arm-poky-linux-gnueabi/gcc/arm-poky-linux-gnueabi/6.4.0
Running clang: --target=arm---gnueabihf -I /home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/beaglebone-lcd//include -I /home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/lib/arm-poky-linux-gnueabi/gcc/arm-poky-linux-gnueabi/6.4.0/include -I /home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/lib/arm-poky-linux-gnueabi/gcc/arm-poky-linux-gnueabi/6.4.0/include-fixed -std=gnu99 -DMONO_GENERATING_OFFSETS -I . -I ./mono -I ./mono/eglib -I . -I ./mono -I ./mono/eglib -DTARGET_ARM -DARM_FPU_VFP -DHAVE_ARMV5 -DHAVE_ARMV6 -DHAVE_ARMV7 -DHOST_LINUX -DMONO_CROSS_COMPILE -DUSE_MONO_CTX -DHAVE_SGEN_GC -DHAVE_MOVING_COLLECTOR ./mono/metadata/metadata-cross-helpers.c

/usr/include/pthread.h:681:6: error: 'regparm' is not valid on this platform

From the error message it seems that clang uses some native x86_64 headers instead of the ARM headers... But mainly I am confused of the command-line options and maybe I provided some path wrongly:

  • --libclang - I assume this is path to the native (x86_64) libclang. I tried different versions, only version 8 works.
  • --monodir and --targetdir - what's the difference? I have only one mono copy which I am trying to compile.
  • --sysroot - I provided a path to the ARM sysroot which I have already compiled.
  • --prefix - I also provided a path to the ARM sysroot. Not sure if this should be absolute or relative path, but relative path simply did not work for me.

Any help would be appreciated.

klasyc avatar Aug 09 '22 11:08 klasyc

After some additional digging I managed to compile the offsets file, but at its beginning it contains #ifndef HAVE_BOEHM_GC macro which prevents my offset definitions from being used. My current configure arguments:

 ./configure --disable-option-checking '--prefix=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr'  '--build=x86_64-linux' '--host=x86_64-linux' '--target=arm-poky-linux-gnueabi' '--exec_prefix=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr' '--bindir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/bin/arm-poky-linux-gnueabi' '--sbindir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/bin/arm-poky-linux-gnueabi' '--libexecdir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/libexec/arm-poky-linux-gnueabi' '--datadir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/share' '--sysconfdir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/etc' '--sharedstatedir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/com' '--localstatedir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/var' '--libdir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/lib/arm-poky-linux-gnueabi' '--includedir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/include' '--oldincludedir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/include' '--infodir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/share/info' '--mandir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/share/man' '--disable-silent-rules' '--disable-dependency-tracking' '--with-libtool-sysroot=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux' 'mono_cv_uscore=no' '--with-sigaltstack=no' '--with-mcs-docs=no' '--with-cross-offsets=/home/ubuntu/yocto/yocto-image-eda/build/tmp/work/x86_64-linux/mono-cross-armv7ahf-neon/6.8.0.123-r0/armv7hf-offsets.h' '--disable-nls' 'build_alias=x86_64-linux' 'host_alias=x86_64-linux' 'target_alias=arm-poky-linux-gnueabi' 'CC=gcc ' 'CFLAGS=-isystem/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/include -O2 -pipe' 'LDFLAGS=-L/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/lib -L/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/lib -Wl,-rpath-link,/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/lib -Wl,-rpath-link,/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/lib -Wl,-rpath,/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/lib -Wl,-rpath,/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/lib -Wl,-O1' 'CPPFLAGS=-isystem/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/include' 'CXX=g++ ' 'CXXFLAGS=-isystem/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/include -O2 -pipe' 'CPP=gcc  -E' '--disable-embed-check' '--with-libgc-threads=pthreads' '--enable-parallel-mark' 'CPPFLAGS_FOR_LIBGC=-Wall -Wunused -Wmissing-declarations -Wpointer-arith -Wno-cast-qual -Wwrite-strings -Wno-switch -Wno-switch-enum -Wno-unused-value -Wno-attributes -isystem/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/include -DGC_LINUX_THREADS -D_GNU_SOURCE -D_REENTRANT -DUSE_MMAP -DUSE_MUNMAP -g -D__ARM_EABI__ -DARM_FPU_VFP=1 -DNO_UNALIGNED_ACCESS -isystem/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/include' 'CFLAGS_FOR_LIBGC=-isystem/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/include -O2 -pipe -Wno-deprecated-declarations -g -mno-tls-direct-seg-refs' --cache-file=/dev/null --srcdir=.

Am I configuring mono incorrectly?

klasyc avatar Aug 10 '22 16:08 klasyc

After some additional digging I managed to compile the offsets file, but at its beginning it contains #ifndef HAVE_BOEHM_GC macro which prevents my offset definitions from being used. My current configure arguments:

 ./configure --disable-option-checking '--prefix=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr'  '--build=x86_64-linux' '--host=x86_64-linux' '--target=arm-poky-linux-gnueabi' '--exec_prefix=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr' '--bindir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/bin/arm-poky-linux-gnueabi' '--sbindir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/bin/arm-poky-linux-gnueabi' '--libexecdir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/libexec/arm-poky-linux-gnueabi' '--datadir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/share' '--sysconfdir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/etc' '--sharedstatedir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/com' '--localstatedir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/var' '--libdir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/lib/arm-poky-linux-gnueabi' '--includedir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/include' '--oldincludedir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/include' '--infodir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/share/info' '--mandir=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/share/man' '--disable-silent-rules' '--disable-dependency-tracking' '--with-libtool-sysroot=/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux' 'mono_cv_uscore=no' '--with-sigaltstack=no' '--with-mcs-docs=no' '--with-cross-offsets=/home/ubuntu/yocto/yocto-image-eda/build/tmp/work/x86_64-linux/mono-cross-armv7ahf-neon/6.8.0.123-r0/armv7hf-offsets.h' '--disable-nls' 'build_alias=x86_64-linux' 'host_alias=x86_64-linux' 'target_alias=arm-poky-linux-gnueabi' 'CC=gcc ' 'CFLAGS=-isystem/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/include -O2 -pipe' 'LDFLAGS=-L/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/lib -L/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/lib -Wl,-rpath-link,/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/lib -Wl,-rpath-link,/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/lib -Wl,-rpath,/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/lib -Wl,-rpath,/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/lib -Wl,-O1' 'CPPFLAGS=-isystem/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/include' 'CXX=g++ ' 'CXXFLAGS=-isystem/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/include -O2 -pipe' 'CPP=gcc  -E' '--disable-embed-check' '--with-libgc-threads=pthreads' '--enable-parallel-mark' 'CPPFLAGS_FOR_LIBGC=-Wall -Wunused -Wmissing-declarations -Wpointer-arith -Wno-cast-qual -Wwrite-strings -Wno-switch -Wno-switch-enum -Wno-unused-value -Wno-attributes -isystem/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/include -DGC_LINUX_THREADS -D_GNU_SOURCE -D_REENTRANT -DUSE_MMAP -DUSE_MUNMAP -g -D__ARM_EABI__ -DARM_FPU_VFP=1 -DNO_UNALIGNED_ACCESS -isystem/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/include' 'CFLAGS_FOR_LIBGC=-isystem/home/ubuntu/yocto/yocto-image-eda/build/tmp/sysroots/x86_64-linux/usr/include -O2 -pipe -Wno-deprecated-declarations -g -mno-tls-direct-seg-refs' --cache-file=/dev/null --srcdir=.

Am I configuring mono incorrectly?

add --disable-boehm

lambdageek avatar Aug 10 '22 18:08 lambdageek

Disabling of boehm helped, but now I have another compile error which seems to be related to the offset file:

| decompose.c: In function ‘mono_decompose_vtype_opts’:
| ../../mono/metadata/abi-details.h:56:42: error: ‘MONO_OFFSET_MonoErrorExternal_init’ undeclared (first use in this function)
|  #define MONO_STRUCT_OFFSET(struct,field) MONO_OFFSET_ ## struct ## _ ## field
|                                           ^
| ir-emit.h:758:29: note: in definition of macro ‘MONO_EMIT_NEW_STORE_MEMBASE_IMM’
|          inst->inst_offset = offset; \
|                              ^
| decompose.c:1255:81: note: in expansion of macro ‘MONO_STRUCT_OFFSET’
|        MONO_EMIT_NEW_STORE_MEMBASE_IMM (cfg, OP_STOREI4_MEMBASE_IMM, dest->dreg, MONO_STRUCT_OFFSET (MonoErrorExternal, init), 0);
|                                                                                  ^
| ../../mono/metadata/abi-details.h:56:42: note: each undeclared identifier is reported only once for each function it appears in
|  #define MONO_STRUCT_OFFSET(struct,field) MONO_OFFSET_ ## struct ## _ ## field
|                                           ^
| ir-emit.h:758:29: note: in definition of macro ‘MONO_EMIT_NEW_STORE_MEMBASE_IMM’
|          inst->inst_offset = offset; \
|                              ^
| decompose.c:1255:81: note: in expansion of macro ‘MONO_STRUCT_OFFSET’
|        MONO_EMIT_NEW_STORE_MEMBASE_IMM (cfg, OP_STOREI4_MEMBASE_IMM, dest->dreg, MONO_STRUCT_OFFSET (MonoErrorExternal, init), 0);

This error is caused by a bug in the offsets-tool.py which does not include offset for MonoErrorExternal datatype. This type is simply missing in the list of types here: https://github.com/mono/mono/blob/ef848cfa83ea16b8afbd5b933968b1838df19505/mono/tools/offsets-tool/offsets-tool.py#L223 and the fix is as simple as adding "MonoErrorExternal" to that list.

klasyc avatar Aug 11 '22 08:08 klasyc

So the AOT cross-compiler seemed to work, but it generates different binaries than the native ARM compiler :( The compiled binaries have even different size. In order to troubleshoot the problem, I tried to generate only assembler files from AOT compiler on ARM (target) and freshly compiled AOT cross-compiler on x64 (build host). Then I made a diff of those two assembler files. Their layout seems to be similar, but there is a huge number of differences in particular bytes.

Now I have two ideas where the problem could be:

  1. Different configure options for target and build host mono. Of course, the set of configure options is different - but are there some options they would affect the AOT compiler output?
  2. My offset file is not correct. Is there any way how to check it? Maybe run the offsets-tool on the target device?

Any ideas?

EDIT: I tried to run the offsets-tool directly on my target device using its native compiler to make sure the offsets-tool running on the build host did not accidentally use wrong header files, but both offset files are identical.

klasyc avatar Aug 31 '22 14:08 klasyc