Using a static library (flang wrapper and f18)

[coti]

Hello,

I am trying to use a static library written in Fortran and called from a program written in Fortran.

• It works fine with a library written in C called from a Fortran program, and with a library written in Fortran called from a C program
• it works fine with a dynamic library
• it works fine with gfortran

My test files are located here. In particular, I use:

$ flang -c libaddf.f90
$ ar cr libaddf.a libaddf.o
$ flang  -o f_and_f f_and_f.f90 libaddf.a 
f_and_f.o: In function `MAIN__':
f18-24f9e.f90:(.text+0x6c): undefined reference to `adddouble_'
f18-24f9e.f90:(.text+0x8c): undefined reference to `addfloat_'
collect2: error: ld returned 1 exit status

However, if I look at the symbols defined in my static library, I can find them:

$ nm libaddf.a

libaddf.o:
000000000000004c T adddouble_
0000000000000000 T addfloat_

If I use a dynamic library, it works fine

$ flang -shared -o libaddf.so libaddf.f90
$ flang  -o f_and_f f_and_f.f90 -L. -laddf    

If I try to link against my static library using the same command-line, the symbols are not found:

$ rm *.so
$ ls *.a
libaddf.a
$ flang  -o f_and_f f_and_f.f90 -L. -laddf
f_and_f.o: In function `MAIN__':
f18-4bff.f90:(.text+0x6c): undefined reference to `adddouble_'
f18-4bff.f90:(.text+0x8c): undefined reference to `addfloat_'
collect2: error: ld returned 1 exit status

I also tried using -fno-underscoring in order to avoid the trailing _:

$ flang -fno-underscoring  -c libaddf.f90
$ ar cr libaddf.a libaddf.o
$ flang -fno-underscoring  -o f_and_f f_and_f.f90 libaddf.a 
f_and_f.o: In function `MAIN__':
f18-da9b.f90:(.text+0x6c): undefined reference to `adddouble'
f18-da9b.f90:(.text+0x8c): undefined reference to `addfloat'
collect2: error: ld returned 1 exit status
$ nm libaddf.a

libaddf.o:
000000000000004c T adddouble
0000000000000000 T addfloat

I have the same errors with f18. If works fine with gfortran:

$ gfortran -c libaddf.f90
$ ar cr libaddf.a libaddf.o
$ gfortran  -o f_and_f f_and_f.f90 libaddf.a 

Is there anything particular to do if I want to work with static libraries?

[sscalpone]

Will 'flang -v' print out the list of commands that are being executed? Maybe something is wrrong?

[coti]

Thank you for your reply. Here is what is being executed:

$ flang -v  -c libaddf.f90
gfortran -v -c -o libaddf.o /tmp/f18-110fb.f90
Using built-in specs.
COLLECT_GCC=gfortran
Target: powerpc64le-unknown-linux-gnu
Configured with: /dev/shm/xxxx/gcc/gcc-7.3.0-build/../gcc-7.3.0/configure --prefix=/usr/local/packages/gcc/7.3.0 --enable-languages=c,c++,fortran
Thread model: posix
gcc version 7.3.0 (GCC) 
COLLECT_GCC_OPTIONS='-v' '-c' '-o' 'libaddf.o'
 /storage/packages/gcc/7.3.0/bin/../libexec/gcc/powerpc64le-unknown-linux-gnu/7.3.0/f951 /tmp/f18-110fb.f90 -quiet -dumpbase f18-110fb.f90 -auxbase-strip libaddf.o -version -fintrinsic-modules-path /storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0/finclude -o /tmp/ccqa5ji1.s
GNU Fortran (GCC) version 7.3.0 (powerpc64le-unknown-linux-gnu)
	compiled by GNU C version 7.3.0, GMP version 6.1.0, MPFR version 3.1.4, MPC version 1.0.3, isl version isl-0.16.1-GMP

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
GNU Fortran2008 (GCC) version 7.3.0 (powerpc64le-unknown-linux-gnu)
	compiled by GNU C version 7.3.0, GMP version 6.1.0, MPFR version 3.1.4, MPC version 1.0.3, isl version isl-0.16.1-GMP

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
COLLECT_GCC_OPTIONS='-v' '-c' '-o' 'libaddf.o'
 as -v -a64 -mppc64 -many -mlittle -o libaddf.o /tmp/ccqa5ji1.s
GNU assembler version 2.27 (ppc64le-redhat-linux) using BFD version version 2.27-34.base.el7_6.3
COMPILER_PATH=/storage/packages/gcc/7.3.0/bin/../libexec/gcc/powerpc64le-unknown-linux-gnu/7.3.0/:/storage/packages/gcc/7.3.0/bin/../libexec/gcc/
LIBRARY_PATH=/storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0/:/storage/packages/gcc/7.3.0/bin/../lib/gcc/:/storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0/../../../../lib64/:/lib/../lib64/:/usr/lib/../lib64/:/storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-v' '-c' '-o' 'libaddf.o'

$ ar cr libaddf.a libaddf.o

$ flang -v  -o f_and_f f_and_f.f90 libaddf.a 
gfortran -v -c -o f_and_f.o /tmp/f18-116c4.f90
Using built-in specs.
COLLECT_GCC=gfortran
Target: powerpc64le-unknown-linux-gnu
Configured with: /dev/shm/xxxx/gcc/gcc-7.3.0-build/../gcc-7.3.0/configure --prefix=/usr/local/packages/gcc/7.3.0 --enable-languages=c,c++,fortran
Thread model: posix
gcc version 7.3.0 (GCC) 
COLLECT_GCC_OPTIONS='-v' '-c' '-o' 'f_and_f.o'
 /storage/packages/gcc/7.3.0/bin/../libexec/gcc/powerpc64le-unknown-linux-gnu/7.3.0/f951 /tmp/f18-116c4.f90 -quiet -dumpbase f18-116c4.f90 -auxbase-strip f_and_f.o -version -fintrinsic-modules-path /storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0/finclude -o /tmp/cco8g6Ca.s
GNU Fortran (GCC) version 7.3.0 (powerpc64le-unknown-linux-gnu)
	compiled by GNU C version 7.3.0, GMP version 6.1.0, MPFR version 3.1.4, MPC version 1.0.3, isl version isl-0.16.1-GMP

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
GNU Fortran2008 (GCC) version 7.3.0 (powerpc64le-unknown-linux-gnu)
	compiled by GNU C version 7.3.0, GMP version 6.1.0, MPFR version 3.1.4, MPC version 1.0.3, isl version isl-0.16.1-GMP

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
COLLECT_GCC_OPTIONS='-v' '-c' '-o' 'f_and_f.o'
 as -v -a64 -mppc64 -many -mlittle -o f_and_f.o /tmp/cco8g6Ca.s
GNU assembler version 2.27 (ppc64le-redhat-linux) using BFD version version 2.27-34.base.el7_6.3
COMPILER_PATH=/storage/packages/gcc/7.3.0/bin/../libexec/gcc/powerpc64le-unknown-linux-gnu/7.3.0/:/storage/packages/gcc/7.3.0/bin/../libexec/gcc/
LIBRARY_PATH=/storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0/:/storage/packages/gcc/7.3.0/bin/../lib/gcc/:/storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0/../../../../lib64/:/lib/../lib64/:/usr/lib/../lib64/:/storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-v' '-c' '-o' 'f_and_f.o'
gfortran -v libaddf.a f_and_f.o -o f_and_f
Driving: gfortran -v libaddf.a f_and_f.o -o f_and_f -l gfortran -l m -shared-libgcc
Using built-in specs.
COLLECT_GCC=gfortran
COLLECT_LTO_WRAPPER=/storage/packages/gcc/7.3.0/bin/../libexec/gcc/powerpc64le-unknown-linux-gnu/7.3.0/lto-wrapper
Target: powerpc64le-unknown-linux-gnu
Configured with: /dev/shm/xxxx/gcc/gcc-7.3.0-build/../gcc-7.3.0/configure --prefix=/usr/local/packages/gcc/7.3.0 --enable-languages=c,c++,fortran
Thread model: posix
gcc version 7.3.0 (GCC) 
Reading specs from /storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0/../../../../lib64/libgfortran.spec
rename spec lib to liborig
COLLECT_GCC_OPTIONS='-v' '-o' 'f_and_f' '-shared-libgcc'
COMPILER_PATH=/storage/packages/gcc/7.3.0/bin/../libexec/gcc/powerpc64le-unknown-linux-gnu/7.3.0/:/storage/packages/gcc/7.3.0/bin/../libexec/gcc/
LIBRARY_PATH=/storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0/:/storage/packages/gcc/7.3.0/bin/../lib/gcc/:/storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0/../../../../lib64/:/lib/../lib64/:/usr/lib/../lib64/:/storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-v' '-o' 'f_and_f' '-shared-libgcc'
 /storage/packages/gcc/7.3.0/bin/../libexec/gcc/powerpc64le-unknown-linux-gnu/7.3.0/collect2 -plugin /storage/packages/gcc/7.3.0/bin/../libexec/gcc/powerpc64le-unknown-linux-gnu/7.3.0/liblto_plugin.so -plugin-opt=/storage/packages/gcc/7.3.0/bin/../libexec/gcc/powerpc64le-unknown-linux-gnu/7.3.0/lto-wrapper -plugin-opt=-fresolution=/tmp/ccU1ruxc.res -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lm -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lgcc --eh-frame-hdr -V -m elf64lppc -dynamic-linker /lib64/ld64.so.2 -o f_and_f /lib/../lib64/crt1.o /lib/../lib64/crti.o /storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0/crtbegin.o -L/storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0 -L/storage/packages/gcc/7.3.0/bin/../lib/gcc -L/storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0/../../.. libaddf.a f_and_f.o -lgfortran -lm -lgcc_s -lgcc -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc /storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0/crtend.o /lib/../lib64/crtn.o
GNU ld version 2.27-34.base.el7_6.3
  Supported emulations:
   elf64lppc
   elf32ppc
   elf32ppclinux
   elf32ppcsim
   elf32lppc
   elf32lppclinux
   elf32lppcsim
   elf64ppc
   elf32_spu
f_and_f.o: In function `MAIN__':
f18-116c4.f90:(.text+0x6c): undefined reference to `adddouble_'
f18-116c4.f90:(.text+0x8c): undefined reference to `addfloat_'
collect2: error: ld returned 1 exit status

[sscalpone]

Look for the following in the output. I think they are in the wrong order. I don't know why.

libaddf.a f_and_f.o

[coti]

Good point.

So in order to see if it was just a reordering issue, I tried:

$ flang f_and_f.f90 libaddf.a -o f_and_f

But the result was the same. So I picked the call to the linker and I added libaddf.a at the end:

/storage/packages/gcc/7.3.0/bin/../libexec/gcc/powerpc64le-unknown-linux-gnu/7.3.0/collect2 \
-plugin /storage/packages/gcc/7.3.0/bin/../libexec/gcc/powerpc64le-unknown-linux-gnu/7.3.0/liblto_plugin.so \
-plugin-opt=/storage/packages/gcc/7.3.0/bin/../libexec/gcc/powerpc64le-unknown-linux-gnu/7.3.0/lto-wrapper \
-plugin-opt=-fresolution=/tmp/ccb5Fo9c.res -plugin-opt=-pass-through=-lgcc_s \
-plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lm -plugin-opt=-pass-through=-lgcc_s \
-plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc_s \
-plugin-opt=-pass-through=-lgcc --eh-frame-hdr -V -m elf64lppc -dynamic-linker /lib64/ld64.so.2 \
-o f_and_f /lib/../lib64/crt1.o /lib/../lib64/crti.o /storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0/crtbegin.o\
 -L/storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0 \
-L/storage/packages/gcc/7.3.0/bin/../lib/gcc -L/storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0/../../../../lib64 \
-L/lib/../lib64 -L/usr/lib/../lib64 \
-L/storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0/../../.. libaddf.a \
f_and_f.o -lgfortran -lm -lgcc_s -lgcc -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc \
/storage/packages/gcc/7.3.0/bin/../lib/gcc/powerpc64le-unknown-linux-gnu/7.3.0/crtend.o \
/lib/../lib64/crtn.o  libaddf.a 

And it worked.

So, is it a bug in how the intermediate command lines are generated?

[coti]

Hi, I wrote a small quick-and-dirty script that grabs the call to the linker and adds the static libraries at the end. It seems to do the trick so far. I have made it available here.

[coti]

Update: in some cases, I have the problem with dynamic libraries too. When I try to compile a Fortran program (here: from the MPICH testsuite), I get

/usr/local/mpich/bin/mpif77 -I. -fcray-pointer   -o allredint8f allredint8f.o ../../f77/util/mtestf.o 
allredint8f.o: In function `MAIN__':
f18-6fb2.f90:(.text+0x4c): undefined reference to `mpi_allreduce_'
../../f77/util/mtestf.o: In function `mtest_init_':
f18-680a.f90:(.text+0x1b): undefined reference to `mpi_initialized_'
f18-680a.f90:(.text+0x31): undefined reference to `mpi_init_'
f18-680a.f90:(.text+0x55): undefined reference to `mpi_comm_rank_'
../../f77/util/mtestf.o: In function `mtest_finalize_':
f18-680a.f90:(.text+0x81): undefined reference to `mpi_comm_rank_'
f18-680a.f90:(.text+0xb9): undefined reference to `mpi_allreduce_'
f18-680a.f90:(.text+0x1e1): undefined reference to `mpi_finalize_'
../../f77/util/mtestf.o: In function `mtestgetintracomm_':
f18-680a.f90:(.text+0x23d): undefined reference to `mpi_comm_dup_'
f18-680a.f90:(.text+0x264): undefined reference to `mpi_comm_size_'
f18-680a.f90:(.text+0x27b): undefined reference to `mpi_comm_rank_'
f18-680a.f90:(.text+0x2b0): undefined reference to `mpi_comm_split_'
../../f77/util/mtestf.o: In function `mtestfreecomm_':
f18-680a.f90:(.text+0x34b): undefined reference to `mpi_comm_free_'
../../f77/util/mtestf.o: In function `mtestprinterror_':
f18-680a.f90:(.text+0x37a): undefined reference to `mpi_error_class_'
f18-680a.f90:(.text+0x39e): undefined reference to `mpi_error_string_'
../../f77/util/mtestf.o: In function `mtestprinterrormsg_':
f18-680a.f90:(.text+0x4c7): undefined reference to `mpi_error_class_'
f18-680a.f90:(.text+0x4eb): undefined reference to `mpi_error_string_'
../../f77/util/mtestf.o: In function `mtestspawnpossible_':
f18-680a.f90:(.text+0x629): undefined reference to `mpi_comm_get_attr_'
f18-680a.f90:(.text+0x670): undefined reference to `mpi_comm_size_'
collect2: error: ld returned 1 exit status

If I look at what mpif77 does, I see:

# /usr/local/mpich/bin/mpif77 -show
/llvm/bin/flang -I/usr/local/mpich/include -I/usr/local/mpich/include -L/usr/local/mpich/lib -lmpifort -lmpi

So I used the trick mentioned above: I looked at the commands executed using flang -v, and I modified the order of the arguments, from:

 /usr/lib/gcc/x86_64-linux-gnu/7/collect2 -plugin /usr/lib/gcc/x86_64-linux-gnu/7/liblto_plugin.so \
-plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/7/lto-wrapper -plugin-opt=-fresolution=/tmp/ccv0L8IC.res \
-plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lgcc \
-plugin-opt=-pass-through=-lquadmath -plugin-opt=-pass-through=-lm \
-plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lgcc \
-plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc_s \
-plugin-opt=-pass-through=-lgcc --build-id --eh-frame-hdr -m elf_x86_64 \
--hash-style=gnu --as-needed -dynamic-linker /lib64/ld-linux-x86-64.so.2 -pie \
-z now -z relro -o allredint8f /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/Scrt1.o \
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/7/crtbeginS.o \
-L/usr/local/mpich/lib -L/usr/lib/gcc/x86_64-linux-gnu/7 -L/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu \
-L/usr/lib/gcc/x86_64-linux-gnu/7/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib \
-L/usr/lib/gcc/x86_64-linux-gnu/7/../../.. -lmpifort -lmpi allredint8f.o ../../f77/util/mtestf.o -lgfortran \
-lm -lgcc_s -lgcc -lquadmath -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc \
/usr/lib/gcc/x86_64-linux-gnu/7/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crtn.o

to

 /usr/lib/gcc/x86_64-linux-gnu/7/collect2 -plugin /usr/lib/gcc/x86_64-linux-gnu/7/liblto_plugin.so \
-plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/7/lto-wrapper -plugin-opt=-fresolution=/tmp/cco2R3e9.res \
-plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lgcc \
-plugin-opt=-pass-through=-lquadmath -plugin-opt=-pass-through=-lm \
-plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lgcc \
-plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc_s \
-plugin-opt=-pass-through=-lgcc --build-id --eh-frame-hdr -m elf_x86_64 \
--hash-style=gnu --as-needed -dynamic-linker /lib64/ld-linux-x86-64.so.2 -pie \
-z now -z relro -o allredint8f /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/Scrt1.o \
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/7/crtbeginS.o \
-L/usr/local/mpich/lib -L/usr/lib/gcc/x86_64-linux-gnu/7 -L/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu \
-L/usr/lib/gcc/x86_64-linux-gnu/7/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib \
-L/usr/lib/gcc/x86_64-linux-gnu/7/../../.. allredint8f.o ../../f77/util/mtestf.o  /usr/lib/gcc/x86_64-linux-gnu/7/crtendS.o \
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crtn.o -lmpifort -lmpi  -lgfortran \
-lm -lgcc_s -lgcc -lquadmath -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc   -lxml2 -lpthread -lrt

and it compiled successfully.

I have updated my wrapper script mentioned above.

[sscalpone]

If you are compiling f18 yourself, you might modify f18.cc to add another list for object files. Then, in CompileFortran add the relos to the object file list instead of the relocatables. Then, call Link if the object file list is not empty and add the object files to the link command before the relocatables.

[coti]

Hi Steve,

Thank you for your help; I have made the modifications you are indicating and it seems to work. I also needed to make a list for the libraries, because they were still at the beginning of the argument list.

With this patch, most MPICH F77 and F90 tests pass. I still have a couple of unrelated compilation errors that I will investigate.

The patch is here: f18_2.patch.txt

[coti]

Update:

• I updated my patch in order to support cases when we compile and link the program with the same command (e.g., flang -o ... ....f)
• while I had my hands in f18.cpp, I fixed f18 -v f18_2.patch.txt