pmodels
3.4.x branch: compile time warnings
Summary stats about warnings
[tkloczko@barrel SPECS]$ cat mpich_warnings.txt|grep \\[-W | sed 's/.*\[//; s/\]//' | sort | uniq -c | sort -nr
157 -Wunused-label
88 -Wlto-type-mismatch
67 -Wunused-dummy-argument
36 -Wc-binding-type
32 -Wmisleading-indentation
29 -Wstringop-overflow=
15 -Wunused-variable
5 -Wunused-but-set-variable
2 -Wunused-function
2 -Wuninitialized
1 -Warray-parameter=
Hmm .. looks like not to many people have been building mpich with LTO.🤔 Without fixing those LTO related warnings LTO optimistation cannot be applied. Full warnings log in attachment mpich_warnings.txt
Most of these warnings are in the Fortran bindings, which is linked separately from the libmpi.so.
What is your compilation process to obtain the mpich_warnings.txt?
I'm building everything using rpm so that log is part of the rpmbuild -ba mpich.spec --quiet output (--quiet redirects stdout to /dev/null so only stderr is in th output)
I think that below script shlud allow you to reproduce LTO optomised build (and all LTO warnings).
Just in case I'm using gcc 11.1.0.
./autogen.sh
CFLAGS='-O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none'
CXXFLAGS='-O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none'
FFLAGS='-O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -I/usr/lib64/gfortran/modules -fallow-argument-mismatch'
FCFLAGS='-O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -I/usr/lib64/gfortran/modules -fallow-argument-mismatch'
LDFLAGS='-Wl,-z,relro -Wl,--as-needed -Wl,--gc-sections -Wl,-z,now -flto=auto -flto-partition=none -fuse-linker-plugin'
CC=/usr/bin/gcc
CXX=/usr/bin/g++
FC=/usr/bin/gfortran
AR=/usr/bin/gcc-ar
NM=/usr/bin/gcc-nm
RANLIB=/usr/bin/gcc-ranlib
export CFLAGS CXXFLAGS FFLAGS FCFLAGS LDFLAGS CC CXX FC AR NM RANLIB
./configure --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --disable-silent-rules --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --runstatedir=/run --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --disable-rpath --disable-static --enable-fc --enable-lib-depend --enable-shared --enable-sharedlibs=gcc --enable-visibility --includedir=/usr/include/mpich --with-device=ch3:nemesis --with-hwloc-prefix=system --without-gnu-ld --with-pm=hydra:gforker
make -j
BTW ..
[tkloczko@barrel mpich-3.4.1]$ find . -name configure.ac
./modules/hwloc/tests/hwloc/embedded/configure.ac
./modules/hwloc/configure.ac
./modules/yaksa/configure.ac
./modules/izem/configure.ac
./modules/libfabric/fabtests/configure.ac
./modules/libfabric/configure.ac
./modules/ucx/configure.ac
./modules/json-c/configure.ac
./maint/fcrosscompile/configure.ac
./maint/configure.ac
./test/mpi/maint/configure.ac
./test/mpi/dtpools/configure.ac
./test/mpi/configure.ac
./src/mpi/romio/configure.ac
./src/mpi/romio/mpl/configure.ac
./src/mpl/configure.ac
./src/pm/hydra2/libhydra/topo/hwloc/hwloc/tests/hwloc/embedded/configure.ac
./src/pm/hydra2/libhydra/topo/hwloc/hwloc/configure.ac
./src/pm/hydra2/configure.ac
./src/pm/hydra2/mpl/configure.ac
./src/pm/hydra/tools/topo/hwloc/hwloc/tests/hwloc/embedded/configure.ac
./src/pm/hydra/tools/topo/hwloc/hwloc/configure.ac
./src/pm/hydra/configure.ac
./src/pm/hydra/mpl/configure.ac
./src/util/logging/rlog/configure.ac
./configure.ac
Lets put on a side all modules/ .. it is any particular reason why mpich uses so many configure.ac files instead just one? Repeating the same detection over and over again is pure waste of time.
Lets put on a side all
modules/.. it is any particular reason why mpich uses so many configure.ac files instead just one? Repeating the same detection over and over again is pure waste of time.
The direction is to converge into a single configure.ac but some parts are more independent than other parts, and merging them will increase the maintenance complexity. You can think of them more like internal "modules". For example, both hydra, romio and "test suite" are being used in external projects.
That said, we are currently wasting a lot of build time in configure and it is in the plan to do something about it.
That said, we are currently wasting a lot of build time in configure and it is in the plan to do something about it.
That is OK .. if you are aware of the possible simplification and someone started working on that that is all wat I need to know. Thank You :) Nevertheless if some of those subdirectories are used in some external projects as submodules all those configure.ac files can stay where they are now. All what in this case needs to be modfied is just main configure.ac and Makefile.am :)
Just in case .. if you will have any problems with reproduce LTO warnings please let me know.
Related to this, I see the BASE_CFLAGS being picked up by default when building UCX and this causes the build to fail with gcc-11 -Werror=array-bounds.
$ rg Werror modules/ucx/config
modules/ucx/config/m4/compiler.m4
12:BASE_CFLAGS="-g -Wall -Werror"
515: [-Werror-implicit-function-declaration],
In file included from /home/jed/src/aur/mpich/src/mpich-3.4.2/build/modules/ucx/../../../modules/ucx/src/ucp/core/ucp_request.inl:17,
from ../../../../../modules/ucx/src/ucp/wireup/wireup.c:17:
../../../../../modules/ucx/src/ucp/wireup/wireup.c: In function ‘ucp_wireup_msg_send’:
/home/jed/src/aur/mpich/src/mpich-3.4.2/build/modules/ucx/../../../modules/ucx/src/ucs/datastruct/mpool.inl:78:10: error: array subscript -1 is outside array bounds of ‘ucp_request_t[1]’ {aka ‘struct u
cp_request[1]’} [-Werror=array-bounds]
78 | mp = elem->mpool;
| ~~~~~^~~~~~~~~~~~~
In file included from /home/jed/src/aur/mpich/src/mpich-3.4.2/build/modules/ucx/../../../modules/ucx/src/ucs/sys/sys.h:19,
from /home/jed/src/aur/mpich/src/mpich-3.4.2/build/modules/ucx/../../../modules/ucx/src/ucs/async/signal.h:13,
from /home/jed/src/aur/mpich/src/mpich-3.4.2/build/modules/ucx/../../../modules/ucx/src/ucs/async/async.h:11,
from /home/jed/src/aur/mpich/src/mpich-3.4.2/build/modules/ucx/../../../modules/ucx/src/ucp/core/ucp_thread.h:14,
from /home/jed/src/aur/mpich/src/mpich-3.4.2/build/modules/ucx/../../../modules/ucx/src/ucp/core/ucp_context.h:14,
from ../../../../../modules/ucx/src/ucp/wireup/wireup.h:11,
from ../../../../../modules/ucx/src/ucp/wireup/wireup.c:11:
/home/jed/src/aur/mpich/src/mpich-3.4.2/build/modules/ucx/../../../modules/ucx/src/ucs/debug/memtrack.h:130:52: note: referencing an object of size 240 allocated by ‘malloc’
130 | #define ucs_malloc(_s, ...) malloc(_s)
| ^~~~~~~~~~
../../../../../modules/ucx/src/ucp/wireup/wireup.c:161:11: note: in expansion of macro ‘ucs_malloc’
161 | req = ucs_malloc(sizeof(*req), "wireup_msg_req");
| ^~~~~~~~~~
In file included from /home/jed/src/aur/mpich/src/mpich-3.4.2/build/modules/ucx/../../../modules/ucx/src/ucp/core/ucp_request.inl:17,
from ../../../../../modules/ucx/src/ucp/wireup/wireup.c:17:
/home/jed/src/aur/mpich/src/mpich-3.4.2/build/modules/ucx/../../../modules/ucx/src/ucs/datastruct/mpool.inl:55:20: error: array subscript -1 is outside array bounds of ‘ucp_request_t[1]’ {aka ‘struct u
cp_request[1]’} [-Werror=array-bounds]
55 | elem->next = mp->freelist;
| ~~~~~~~~~~~^~~~~~~~~~~~~~
In file included from /home/jed/src/aur/mpich/src/mpich-3.4.2/build/modules/ucx/../../../modules/ucx/src/ucs/sys/sys.h:19,
from /home/jed/src/aur/mpich/src/mpich-3.4.2/build/modules/ucx/../../../modules/ucx/src/ucs/async/signal.h:13,
from /home/jed/src/aur/mpich/src/mpich-3.4.2/build/modules/ucx/../../../modules/ucx/src/ucs/async/async.h:11,
from /home/jed/src/aur/mpich/src/mpich-3.4.2/build/modules/ucx/../../../modules/ucx/src/ucp/core/ucp_thread.h:14,
from /home/jed/src/aur/mpich/src/mpich-3.4.2/build/modules/ucx/../../../modules/ucx/src/ucp/core/ucp_context.h:14,
from ../../../../../modules/ucx/src/ucp/wireup/wireup.h:11,
from ../../../../../modules/ucx/src/ucp/wireup/wireup.c:11:
/home/jed/src/aur/mpich/src/mpich-3.4.2/build/modules/ucx/../../../modules/ucx/src/ucs/debug/memtrack.h:130:52: note: referencing an object of size 240 allocated by ‘malloc’
130 | #define ucs_malloc(_s, ...) malloc(_s)
| ^~~~~~~~~~
../../../../../modules/ucx/src/ucp/wireup/wireup.c:161:11: note: in expansion of macro ‘ucs_malloc’
161 | req = ucs_malloc(sizeof(*req), "wireup_msg_req");
| ^~~~~~~~~~
cc1: all warnings being treated as errors
Note that adding -Wno-error=array-bounds to MPICHLIB_CFLAGS does not cause it to be propagated to UCX.
This is currently an issue for Arch Linux packaging of 3.4.1 and 3.4.2. https://aur.archlinux.org/packages/mpich/
