Compilation Error in PIRA: Incorrect Field Usage in MCGBaseInfo.h

[Masterwater-y]

Dear PIRA Development Team,

I encountered a compilation error while attempting to build PIRA. The error message indicates an issue with the usage of undefined fields in the MCGFileFormatVersion and MCGGeneratorVersionInfo classes within the MCGBaseInfo.h header file.

I execute ./build_submodules.sh and it seems encounter error in buiding MetaCG

The error shows below

44%] Building CXX object graph/CMakeFiles/metacg.dir/src/MCGWriter.cpp.o cd /home/yhl/pira/extern/src/metacg/build/graph && /opt/rh/devtoolset-8/root/usr/bin/g++ -DFMT_SHARED -DJSON_DIAGNOSTICS=0 -DJSON_USE_IMPLICIT_CONVERSIONS=1 -DSPDLOG_COMPILED_LIB -DSPDLOG_SHARED_LIB -Dmetacg_EXPORTS -I/home/yhl/pira/extern/src/metacg/graph/include -I/home/yhl/pira/extern/src/metacg/pgis/lib/include -I/home/yhl/pira/extern/src/metacg/build -I/home/yhl/pira/extern/src/metacg/build/_deps/json-src/single_include -I/home/yhl/pira/extern/src/metacg/build/_deps/spdlog-src/include -isystem /home/yhl/pira/extern/install/extrap/include -g -std=c++17 -fPIC -MD -MT graph/CMakeFiles/metacg.dir/src/MCGWriter.cpp.o -MF CMakeFiles/metacg.dir/src/MCGWriter.cpp.o.d -o CMakeFiles/metacg.dir/src/MCGWriter.cpp.o -c /home/yhl/pira/extern/src/metacg/graph/src/MCGWriter.cpp In file included from /usr/include/sys/types.h:222, from /usr/include/stdlib.h:314, from /opt/rh/devtoolset-8/root/usr/include/c++/8/cstdlib:75, from /opt/rh/devtoolset-8/root/usr/include/c++/8/ext/string_conversions.h:41, from /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/basic_string.h:6400, from /opt/rh/devtoolset-8/root/usr/include/c++/8/string:52, from /opt/rh/devtoolset-8/root/usr/include/c++/8/stdexcept:39, from /opt/rh/devtoolset-8/root/usr/include/c++/8/array:39, from /opt/rh/devtoolset-8/root/usr/include/c++/8/tuple:39, from /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/unique_ptr.h:37, from /opt/rh/devtoolset-8/root/usr/include/c++/8/memory:80, from /home/yhl/pira/extern/src/metacg/graph/include/CgNodePtr.h:9, from /home/yhl/pira/extern/src/metacg/graph/include/MetaData.h:9, from /home/yhl/pira/extern/src/metacg/graph/include/CgNode.h:11, from /home/yhl/pira/extern/src/metacg/graph/include/Callgraph.h:9, from /home/yhl/pira/extern/src/metacg/graph/include/MCGWriter.h:10, from /home/yhl/pira/extern/src/metacg/graph/src/MCGWriter.cpp:7: /home/yhl/pira/extern/src/metacg/graph/include/MCGBaseInfo.h: In constructor ‘metacg::MCGFileFormatVersion::MCGFileFormatVersion(int, int)’: /home/yhl/pira/extern/src/metacg/graph/include/MCGBaseInfo.h:17:48: error: class ‘metacg::MCGFileFormatVersion’ does not have any field named ‘gnu_dev_major’ MCGFileFormatVersion(int major, int minor) : major(major), minor(minor) {} ^~~~~ /home/yhl/pira/extern/src/metacg/graph/include/MCGBaseInfo.h:17:62: error: class ‘metacg::MCGFileFormatVersion’ does not have any field named ‘gnu_dev_minor’ MCGFileFormatVersion(int major, int minor) : major(major), minor(minor) {} ^~~~~ /home/yhl/pira/extern/src/metacg/graph/include/MCGBaseInfo.h: In constructor ‘metacg::MCGGeneratorVersionInfo::MCGGeneratorVersionInfo(std::string&, int, int, std::string)’: /home/yhl/pira/extern/src/metacg/graph/include/MCGBaseInfo.h:37:21: error: class ‘metacg::MCGGeneratorVersionInfo’ does not have any field named ‘gnu_dev_major’ : name(name), major(major), minor(minor), sha(std::move(gitSHA)) {} ^~~~~ /home/yhl/pira/extern/src/metacg/graph/include/MCGBaseInfo.h:37:35: error: class ‘metacg::MCGGeneratorVersionInfo’ does not have any field named ‘gnu_dev_minor’ : name(name), major(major), minor(minor), sha(std::move(gitSHA)) {} ^~~~~ In file included from /usr/include/sys/types.h:222, from /usr/include/stdlib.h:314, from /opt/rh/devtoolset-8/root/usr/include/c++/8/cstdlib:75, from /opt/rh/devtoolset-8/root/usr/include/c++/8/ext/string_conversions.h:41, from /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/basic_string.h:6400, from /opt/rh/devtoolset-8/root/usr/include/c++/8/string:52, from /opt/rh/devtoolset-8/root/usr/include/c++/8/stdexcept:39, from /opt/rh/devtoolset-8/root/usr/include/c++/8/array:39, from /opt/rh/devtoolset-8/root/usr/include/c++/8/tuple:39, from /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/unique_ptr.h:37, from /opt/rh/devtoolset-8/root/usr/include/c++/8/memory:80, from /home/yhl/pira/extern/src/metacg/graph/include/CgNodePtr.h:9, from /home/yhl/pira/extern/src/metacg/graph/include/MetaData.h:9, from /home/yhl/pira/extern/src/metacg/graph/include/CgNode.h:11, from /home/yhl/pira/extern/src/metacg/graph/include/Callgraph.h:9, from /home/yhl/pira/extern/src/metacg/graph/include/MCGManager.h:9, from /home/yhl/pira/extern/src/metacg/graph/include/MCGReader.h:9, from /home/yhl/pira/extern/src/metacg/graph/src/MCGReader.cpp:6: /home/yhl/pira/extern/src/metacg/graph/include/MCGBaseInfo.h: In constructor ‘metacg::MCGFileFormatVersion::MCGFileFormatVersion(int, int)’: /home/yhl/pira/extern/src/metacg/graph/include/MCGBaseInfo.h:17:48: error: class ‘metacg::MCGFileFormatVersion’ does not have any field named ‘gnu_dev_major’ MCGFileFormatVersion(int major, int minor) : major(major), minor(minor) {} ^~~~~ /home/yhl/pira/extern/src/metacg/graph/include/MCGBaseInfo.h:17:62: error: class ‘metacg::MCGFileFormatVersion’ does not have any field named ‘gnu_dev_minor’ MCGFileFormatVersion(int major, int minor) : major(major), minor(minor) {} ^~~~~ /home/yhl/pira/extern/src/metacg/graph/include/MCGBaseInfo.h: In constructor ‘metacg::MCGGeneratorVersionInfo::MCGGeneratorVersionInfo(std::string&, int, int, std::string)’: /home/yhl/pira/extern/src/metacg/graph/include/MCGBaseInfo.h:37:21: error: class ‘metacg::MCGGeneratorVersionInfo’ does not have any field named ‘gnu_dev_major’ : name(name), major(major), minor(minor), sha(std::move(gitSHA)) {} ^~~~~ /home/yhl/pira/extern/src/metacg/graph/include/MCGBaseInfo.h:37:35: error: class ‘metacg::MCGGeneratorVersionInfo’ does not have any field named ‘gnu_dev_minor’ : name(name), major(major), minor(minor), sha(std::move(gitSHA)) {}

[jplehr]

Thank you for the report. We're gonna look into it.

Edit: My bad, I forgot to ask. Are you using the main branch or the devel branch?

[jplehr]

It appears that in some versions of GCC the identifiers major and minor were macros. We have pushed a fix to MetaCG that works for us. Please let me know if this fixes your compilation issue as well.

Edit: If you are building MetaCG as part of PIRA, this fix will not have reached you, unfortunately. PIRA currently tracks version 0.4 from MetaCG. We are working on bumping the version to a newer MetaCG release.

[Masterwater-y]

@jplehr Thanks for fixing the issue. I modify the 347th row of build_submodules.sh as below to get the latest code, and it works, I can finish running build_submodules.sh without errors

git clone https://github.com/tudasc/MetaCG.git metacg

But some new problems occur when I try to run the example GameOfLife, I execute ./run.sh, and errors come out as below. I have checked /extern/install/metacg/bin, cgc, cgmerge seems not generated successfully,

----- Build GameOfLife / build call graph ----- SerialGoL.cpp: In member function ‘const int GameOfLife::getNumLiveNeighbors(int, int) const’: SerialGoL.cpp:94:45: warning: comparison between signed and unsigned integer expressions [-Wsign-compare] if(neighbor >= 0 && neighbor < gridA.size()){ ^ ./run.sh: line 37: cgc: command not found ./run.sh: line 38: cgc: command not found ./run.sh: line 40: cgmerge: command not found ----- Running Pira ----- Using /root/.local/share/pira for runtime files [INFO in '/home/yhl/pira/lib/Pira.py' in 'main' line 168]: Running PIRA in compiletime filtering with configuration /home/yhl/pira/test/integration/GameOfLife/gol_config.json [WARNING in '/home/yhl/pira/lib/Pira.py' in 'process_args_for_extrap' line 147]: At least 5 repetitions are recommended for Extra-P modelling. [INFO in '/home/yhl/pira/lib/Checker.py' in 'check_configfile_v2' line 84]: Creating Cubes-Directory /root/.local/share/pira/gol_cubes [INFO in '/home/yhl/pira/lib/Pira.py' in 'main' line 190]: Running the local case [INFO in '/home/yhl/pira/lib/Pira.py' in 'execute_with_config' line 40]: Building vanilla version for baseline measurements [INFO in '/home/yhl/pira/lib/Pira.py' in 'execute_with_config' line 46]: Running baseline measurements [INFO in '/home/yhl/pira/lib/Pira.py' in 'execute_with_config' line 57]: Running instrumentation iteration 0 [ERROR in '/home/yhl/pira/lib/Utility.py' in 'shell' line 346]: Utility::shell: Caught Exception Command 'pgis_pira --scorep-out --static /home/yhl/pira/test/integration/GameOfLife/../../../extern/install/metacg/bin//gol_ct.mcg' returned non-zero exit status 127. [ERROR in '/home/yhl/pira/lib/Analyzer.py' in 'analyze_local' line 111]: Utility::shell: Running command pgis_pira --scorep-out --static /home/yhl/pira/test/integration/GameOfLife/../../../extern/install/metacg/bin//gol_ct.mcg did not succeed [ERROR in '/home/yhl/pira/lib/Pira.py' in 'execute_with_config' line 111]: Pira::execute_with_config: Problem during preparation of run. Message: Problem in Analyzer [ERROR in '/home/yhl/pira/lib/Pira.py' in 'main' line 256]: Runner.run caught exception. Message: Problem in Analyzer Error opening instrumentation file for iteration 0: [Errno 2] No such file or directory: '../../../extern/install/metacg/bin/out/instrumented-gol_ct_it-0.txt' Error opening instrumentation file for iteration 1: [Errno 2] No such file or directory: '../../../extern/install/metacg/bin/out/instrumented-gol_ct_it-1.txt' Error opening instrumentation file for iteration 2: [Errno 2] No such file or directory: '../../../extern/install/metacg/bin/out/instrumented-gol_ct_it-2.txt'

(py38) [root@localhost bin]# ls gol_ct.mcg out pgis_cfg_gol.json (py38) [root@localhost bin]# pwd /home/yhl/pira/extern/install/metacg/bin

[jplehr]

Cool to hear that is works and fixes the compiler error.

We are working internally to move the tracked MetaCG up to the latest release and have it pass our CI. Once that is done, we'll have another PIRA release. We did quite a bit of work in MetaCG in the meantime, so previous assumptions in PIRA may no longer hold.

[jplehr]

I worked on this yesterday. After bumping the version, you need to add two additional CMake flags to the MetaCG build in resources/build_submodules.sh:

-DMETACG_BUILD_PGIS=ON \
-DMETACG_BUILD_CGCOLLECTOR=ON

This should build/install all components that PIRA assumes and unblock you there. However, we see some test failures in our CI and look into them.

[Masterwater-y]

I try it and some new errors occur as below while building metacg and I checked that it can build successfully without errors if i don't add those two additional flags

I modified ./build_submodules as

356 check_directory_or_file_exists $extsourcedir/metacg/build 357 if [ $? -ne 0 ]; then 358 echo_configuring "$metacg_component_name" 359 cmake -B build -S . -DCMAKE_BUILD_TYPE=Debug
360 -DCMAKE_INSTALL_PREFIX=$extinstalldir/metacg
361 -DCUBE_LIB=$cube_install_dir/lib
362 -DCUBE_INCLUDE=$cube_install_dir/include/cubelib
363 -DEXTRAP_INCLUDE=$extrap_install_dir/include
364 -DEXTRAP_LIB=$extrap_install_dir/lib
365 -DSPDLOG_BUILD_SHARED=ON
366 -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON
367 -DMETACG_BUILD_PGIS=ON
368 -DMETACG_BUILD_CGCOLLECTOR=ON >${allOutputTo} 2>&1

new errors:

n file included from /home/yhl/pira1229/extern/src/metacg/pgis/lib/src/ExtrapConnection.cpp:7: /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/ExtrapConnection.h:19:10: fatal error: EXTRAP_CubeFileReader.hpp: No such file or directory 19 | #include "EXTRAP_CubeFileReader.hpp" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. gmake[2]: *** [pgis/lib/CMakeFiles/pgis.dir/src/ExtrapConnection.cpp.o] Error 1 gmake[2]: *** Waiting for unfinished jobs.... In file included from /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/MetaData/CgNodeMetaData.h:19, from /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/loadImbalance/LIMetaData.h:13, from /home/yhl/pira1229/extern/src/metacg/pgis/lib/src/CgHelper.cpp:9: /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/ExtrapConnection.h:19:10: fatal error: EXTRAP_CubeFileReader.hpp: No such file or directory 19 | #include "EXTRAP_CubeFileReader.hpp" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. gmake[2]: *** [pgis/lib/CMakeFiles/pgis.dir/src/CgHelper.cpp.o] Error 1 In file included from /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/MetaData/CgNodeMetaData.h:19, from /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/IPCGEstimatorPhase.h:11, from /home/yhl/pira1229/extern/src/metacg/pgis/lib/src/IPCGEstimatorPhase.cpp:10: /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/ExtrapConnection.h:19:10: fatal error: EXTRAP_CubeFileReader.hpp: No such file or directory 19 | #include "EXTRAP_CubeFileReader.hpp" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. gmake[2]: *** [pgis/lib/CMakeFiles/pgis.dir/src/IPCGEstimatorPhase.cpp.o] Error 1 In file included from /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/MetaData/CgNodeMetaData.h:19, from /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/ExtrapEstimatorPhase.h:15, from /home/yhl/pira1229/extern/src/metacg/pgis/lib/src/ExtrapEstimatorPhase.cpp:10: /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/ExtrapConnection.h:19:10: fatal error: EXTRAP_CubeFileReader.hpp: No such file or directory 19 | #include "EXTRAP_CubeFileReader.hpp" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. gmake[2]: *** [pgis/lib/CMakeFiles/pgis.dir/src/ExtrapEstimatorPhase.cpp.o] Error 1 In file included from /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/MetaData/CgNodeMetaData.h:19, from /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/loadImbalance/LIConfig.h:11, from /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/config/ParameterConfig.h:12, from /home/yhl/pira1229/extern/src/metacg/pgis/lib/src/config/ParameterConfig.cpp:7: /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/ExtrapConnection.h:19:10: fatal error: EXTRAP_CubeFileReader.hpp: No such file or directory 19 | #include "EXTRAP_CubeFileReader.hpp" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. gmake[2]: *** [pgis/lib/CMakeFiles/pgis.dir/src/config/ParameterConfig.cpp.o] Error 1 In file included from /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/MetaData/CgNodeMetaData.h:19, from /home/yhl/pira1229/extern/src/metacg/pgis/lib/src/loadImbalance/metric/AbstractMetric.cpp:8: /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/ExtrapConnection.h:19:10: fatal error: EXTRAP_CubeFileReader.hpp: No such file or directory 19 | #include "EXTRAP_CubeFileReader.hpp" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. gmake[2]: *** [pgis/lib/CMakeFiles/pgis.dir/src/loadImbalance/metric/AbstractMetric.cpp.o] Error 1 In file included from /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/MetaData/CgNodeMetaData.h:19, from /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/loadImbalance/LIMetaData.h:13, from /home/yhl/pira1229/extern/src/metacg/pgis/lib/src/LegacyMCGReader.cpp:10: /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/ExtrapConnection.h:19:10: fatal error: EXTRAP_CubeFileReader.hpp: No such file or directory 19 | #include "EXTRAP_CubeFileReader.hpp" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. In file included from /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/MetaData/CgNodeMetaData.h:19, from /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/loadImbalance/LIMetaData.h:13, from /home/yhl/pira1229/extern/src/metacg/pgis/lib/src/loadImbalance/LIMetaData.cpp:7: /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/ExtrapConnection.h:19:10: fatal error: EXTRAP_CubeFileReader.hpp: No such file or directory 19 | #include "EXTRAP_CubeFileReader.hpp" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. gmake[2]: *** [pgis/lib/CMakeFiles/pgis.dir/src/LegacyMCGReader.cpp.o] Error 1 gmake[2]: *** [pgis/lib/CMakeFiles/pgis.dir/src/loadImbalance/LIMetaData.cpp.o] Error 1 In file included from /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/PiraMCGProcessor.h:17, from /home/yhl/pira1229/extern/src/metacg/pgis/lib/src/PiraMCGProcessor.cpp:7: /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/ExtrapConnection.h:19:10: fatal error: EXTRAP_CubeFileReader.hpp: No such file or directory 19 | #include "EXTRAP_CubeFileReader.hpp" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. gmake[2]: *** [pgis/lib/CMakeFiles/pgis.dir/src/PiraMCGProcessor.cpp.o] Error 1 In file included from /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/MetaData/CgNodeMetaData.h:19, from /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/loadImbalance/LIConfig.h:11, from /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/loadImbalance/LIEstimatorPhase.h:11, from /home/yhl/pira1229/extern/src/metacg/pgis/lib/src/loadImbalance/LIEstimatorPhase.cpp:7: /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/ExtrapConnection.h:19:10: fatal error: EXTRAP_CubeFileReader.hpp: No such file or directory 19 | #include "EXTRAP_CubeFileReader.hpp" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. gmake[2]: *** [pgis/lib/CMakeFiles/pgis.dir/src/loadImbalance/LIEstimatorPhase.cpp.o] Error 1 In file included from /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/MetaData/CgNodeMetaData.h:19, from /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/CubeReader.h:10, from /home/yhl/pira1229/extern/src/metacg/pgis/lib/src/CubeReader.cpp:7: /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/ExtrapConnection.h:19:10: fatal error: EXTRAP_CubeFileReader.hpp: No such file or directory 19 | #include "EXTRAP_CubeFileReader.hpp" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. gmake[2]: *** [pgis/lib/CMakeFiles/pgis.dir/src/CubeReader.cpp.o] Error 1 In file included from /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/MetaData/CgNodeMetaData.h:19, from /home/yhl/pira1229/extern/src/metacg/pgis/lib/src/EstimatorPhase.cpp:8: /home/yhl/pira1229/extern/src/metacg/pgis/lib/include/ExtrapConnection.h:19:10: fatal error: EXTRAP_CubeFileReader.hpp: No such file or directory 19 | #include "EXTRAP_CubeFileReader.hpp" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. gmake[2]: *** [pgis/lib/CMakeFiles/pgis.dir/src/EstimatorPhase.cpp.o] Error 1 gmake[2]: Leaving directory /home/yhl/pira1229/extern/src/metacg/build' gmake[1]: *** [pgis/lib/CMakeFiles/pgis.dir/all] Error 2 gmake[1]: *** Waiting for unfinished jobs.... [ 73%] Linking CXX executable libtests cd /home/yhl/pira1229/extern/src/metacg/build/graph/test/unit && /home/yhl/pkg/cmake-3.21.3-linux-x86_64/bin/cmake -E cmake_link_script CMakeFiles/libtests.dir/link.txt --verbose=1 /opt/rh/devtoolset-9/root/usr/bin/g++ -g CMakeFiles/libtests.dir/DotIOTest.cpp.o CMakeFiles/libtests.dir/MCGManagerTest.cpp.o CMakeFiles/libtests.dir/MCGReaderTest.cpp.o CMakeFiles/libtests.dir/MCGWriterTest.cpp.o CMakeFiles/libtests.dir/ReachabilityAnalysisTest.cpp.o CMakeFiles/libtests.dir/UtilTest.cpp.o -o libtests -Wl,-rpath,/home/yhl/pira1229/extern/src/metacg/build/graph:/home/yhl/pira1229/extern/src/metacg/build/_deps/spdlog-build ../../../lib/libgtest_main.a ../../libmetacg.so.0.6.0 ../../../_deps/spdlog-build/libspdlogd.so.1.8.2 ../../../lib/libgtest.a -lpthread gmake[2]: Leaving directory /home/yhl/pira1229/extern/src/metacg/build' [ 73%] Built target libtests [ 74%] Linking CXX shared library libcollector.so cd /home/yhl/pira1229/extern/src/metacg/build/cgcollector/lib && /home/yhl/pkg/cmake-3.21.3-linux-x86_64/bin/cmake -E cmake_link_script CMakeFiles/collector.dir/link.txt --verbose=1 /opt/rh/devtoolset-9/root/usr/bin/g++ -fPIC -g -shared -Wl,-soname,libcollector.so -o libcollector.so CMakeFiles/collector.dir/src/AAReferenceVisitor.cpp.o CMakeFiles/collector.dir/src/AAUSR.cpp.o CMakeFiles/collector.dir/src/AliasAnalysis.cpp.o CMakeFiles/collector.dir/src/CallGraph.cpp.o CMakeFiles/collector.dir/src/CallgraphToJSON.cpp.o CMakeFiles/collector.dir/src/GlobalCallDepth.cpp.o CMakeFiles/collector.dir/src/helper/ASTHelper.cpp.o CMakeFiles/collector.dir/src/helper/common.cpp.o CMakeFiles/collector.dir/src/JSONManager.cpp.o -Wl,-rpath,/opt/rh/llvm-toolset-10.0/root/usr/lib64: -lclangTooling -lclangFrontend -lclangSerialization -lclangAST -lclangBasic -lclangIndex /opt/rh/llvm-toolset-10.0/root/usr/lib64/libLLVM-10.so /opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: cannot find -lclangTooling /opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: cannot find -lclangFrontend /opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: cannot find -lclangSerialization /opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: cannot find -lclangAST /opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: cannot find -lclangBasic /opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: cannot find -lclangIndex collect2: error: ld returned 1 exit status gmake[2]: *** [cgcollector/lib/libcollector.so] Error 1 gmake[2]: Leaving directory /home/yhl/pira1229/extern/src/metacg/build' gmake[1]: *** [cgcollector/lib/CMakeFiles/collector.dir/all] Error 2 gmake[1]: Leaving directory /home/yhl/pira1229/extern/src/metacg/build' gmake: *** [all] Error 2

[jplehr]

Just to clarify this for me: This is inside the container or on the host system? From the paths etc, I think this is on the host system, right?

The build_submodules.sh script may have not built all dependencies. We had some trouble with it every now and then, but focused on other aspects.

If I recall correctly then the issue comes from that the script only tests if build directories exist, but nothing more. So, in case you had a failed build, the script may get confused and think everything is there when it's in fact not the case. It also seems that it has some trouble finding the LLVM libraries.

[Masterwater-y]

It is on the host system, CentOS7 I have repeatedly confirmed the build process, and ensured that both score-p and extrap were built correctly by outputting logs to the console. However, I still encounter the following errors.

The build_submodule file downloads these two components using the following command. I wonder if it is a version issue with extrap.

git clone -b v0.2 --single-branch https://github.com/jplehr/score-p-v6.git scorep-mod wget http://apps.fz-juelich.de/scalasca/releases/extra-p/extrap-3.0.tar.gz

In file included from /home/yhl/pira/extern/src/metacg/pgis/lib/src/ExtrapConnection.cpp:7: /home/yhl/pira/extern/src/metacg/pgis/lib/include/ExtrapConnection.h:19:10: fatal error: EXTRAP_CubeFileReader.hpp: No such file or directory 19 | #include "EXTRAP_CubeFileReader.hpp" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. gmake[2]: *** [pgis/lib/CMakeFiles/pgis.dir/src/ExtrapConnection.cpp.o] Error 1 gmake[2]: *** Waiting for unfinished jobs.... In file included from /home/yhl/pira/extern/src/metacg/pgis/lib/include/PiraMCGProcessor.h:17, from /home/yhl/pira/extern/src/metacg/pgis/lib/src/PiraMCGProcessor.cpp:7: /home/yhl/pira/extern/src/metacg/pgis/lib/include/ExtrapConnection.h:19:10: fatal error: EXTRAP_CubeFileReader.hpp: No such file or directory 19 | #include "EXTRAP_CubeFileReader.hpp" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. In file included from /home/yhl/pira/extern/src/metacg/pgis/lib/include/MetaData/CgNodeMetaData.h:19, from /home/yhl/pira/extern/src/metacg/pgis/lib/src/loadImbalance/metric/AbstractMetric.cpp:8: /home/yhl/pira/extern/src/metacg/pgis/lib/include/ExtrapConnection.h:19:10: fatal error: EXTRAP_CubeFileReader.hpp: No such file or directory 19 | #include "EXTRAP_CubeFileReader.hpp" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. gmake[2]: *** [pgis/lib/CMakeFiles/pgis.dir/src/PiraMCGProcessor.cpp.o] Error 1 gmake[2]: *** [pgis/lib/CMakeFiles/pgis.dir/src/loadImbalance/metric/AbstractMetric.cpp.o] Error 1 In file included from /home/yhl/pira/extern/src/metacg/pgis/lib/include/MetaData/CgNodeMetaData.h:19, from /home/yhl/pira/extern/src/metacg/pgis/lib/src/EstimatorPhase.cpp:8: /home/yhl/pira/extern/src/metacg/pgis/lib/include/ExtrapConnection.h:19:10: fatal error: EXTRAP_CubeFileReader.hpp: No such file or directory 19 | #include "EXTRAP_CubeFileReader.hpp" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated.

The log of building extrap

[PIRA] Configuring Extra-P [PIRA] Found Python.h at /root/anaconda3/envs/py38/bin/../include/python3.8 [PIRA] Building Extra-P make all-am make[1]: Entering directory /home/yhl/pira/extern/src/extrap/extrap-3.0/build' CXX libextrap_la-EXTRAP_Arithmetics.lo CXX libextrap_la-EXTRAP_Callpath.lo CXX libextrap_la-EXTRAP_CubeMapping.lo CXX libextrap_la-EXTRAP_Coordinate.lo CXX libextrap_la-EXTRAP_DataPoint.lo CXX libextrap_la-EXTRAP_CompoundTerm.lo CXX libextrap_la-EXTRAP_CubeFileReader.lo CXX libextrap_la-EXTRAP_Experiment.lo CXX libextrap_la-EXTRAP_ExperimentPoint.lo CXX libextrap_la-EXTRAP_Function.lo CXX libextrap_la-EXTRAP_Fraction.lo CXX libextrap_la-EXTRAP_IncrementalPoint.lo CXX libextrap_la-EXTRAP_IoHelper.lo CXX libextrap_la-EXTRAP_MessageStream.lo CXX libextrap_la-EXTRAP_Model.lo CXX libextrap_la-EXTRAP_Metric.lo CXX libextrap_la-EXTRAP_ModelGenerator.lo CXX libextrap_la-EXTRAP_ModelComment.lo CXX libextrap_la-EXTRAP_Region.lo CXX libextrap_la-EXTRAP_SimpleTerm.lo CXX libextrap_la-EXTRAP_Parameter.lo CXX libextrap_la-EXTRAP_SingleParameterHypothesis.lo CXX libextrap_la-EXTRAP_SingleParameterFunction.lo CXX libextrap_la-EXTRAP_SingleParameterModelGenerator.lo CXX libextrap_la-EXTRAP_SingleParameterFunctionModeler.lo CXX libextrap_la-EXTRAP_SingleParameterExhaustiveModelGenerator.lo CXX libextrap_la-EXTRAP_SingleParameterExhaustiveFunctionModeler.lo CXX libextrap_la-EXTRAP_SingleParameterSimpleModelGenerator.lo CXX libextrap_la-EXTRAP_SingleParameterSimpleFunctionModeler.lo CXX libextrap_la-EXTRAP_SingleParameterRefiningFunctionModeler.lo CXX libextrap_la-EXTRAP_SingleParameterRefiningModelGenerator.lo CXX libextrap_la-EXTRAP_MultiParameterFunctionModeler.lo CXX libextrap_la-EXTRAP_MultiParameterTerm.lo CXX libextrap_la-EXTRAP_MultiParameterFunction.lo CXX libextrap_la-EXTRAP_MultiParameterHypothesis.lo CXX libextrap_la-EXTRAP_MultiParameterModelGenerator.lo CXX libextrap_la-EXTRAP_MultiParameterSimpleModelGenerator.lo CXX libextrap_la-EXTRAP_Utilities.lo CXX libextrap_la-EXTRAP_MultiParameterSimpleFunctionModeler.lo CXX libextrap_python_la-EXTRAP_Interface_wrap.lo CXXLD libextrap.la ../src/python_wrapper/c++/EXTRAP_Interface_wrap.cpp: In function ‘void SWIG_Python_addvarlink(PyObject*, char*, PyObject* (*)(), int (*)(PyObject*))’: ../src/python_wrapper/c++/EXTRAP_Interface_wrap.cpp:44846:16: warning: ‘char* strncpy(char*, const char*, size_t)’ specified bound depends on the length of the source argument [-Wstringop-overflow=] 44846 | strncpy(gv->name,name,size); | ~~~~~~~^~~~~~~~~~~~~~~~~~~~ ../src/python_wrapper/c++/EXTRAP_Interface_wrap.cpp:44843:27: note: length computed here 44843 | size_t size = strlen(name)+1; | ~~~~~~^~~~~~ CXXLD libextrap_python.la make[1]: Leaving directory /home/yhl/pira/extern/src/extrap/extrap-3.0/build' make[1]: Entering directory `/home/yhl/pira/extern/src/extrap/extrap-3.0/build' /usr/bin/mkdir -p '/home/yhl/pira/resources/../extern/install/extrap/lib' /bin/sh ./libtool --mode=install /usr/bin/install -c libextrap.la libextrap_python.la '/home/yhl/pira/resources/../extern/install/extrap/lib' libtool: install: /usr/bin/install -c .libs/libextrap.so.0.0.0 /home/yhl/pira/resources/../extern/install/extrap/lib/libextrap.so.0.0.0 libtool: install: (cd /home/yhl/pira/resources/../extern/install/extrap/lib && { ln -s -f libextrap.so.0.0.0 libextrap.so.0 || { rm -f libextrap.so.0 && ln -s libextrap.so.0.0.0 libextrap.so.0; }; }) libtool: install: (cd /home/yhl/pira/resources/../extern/install/extrap/lib && { ln -s -f libextrap.so.0.0.0 libextrap.so || { rm -f libextrap.so && ln -s libextrap.so.0.0.0 libextrap.so; }; }) libtool: install: /usr/bin/install -c .libs/libextrap.lai /home/yhl/pira/resources/../extern/install/extrap/lib/libextrap.la libtool: warning: relinking 'libextrap_python.la' libtool: install: (cd /home/yhl/pira/extern/src/extrap/extrap-3.0/build; /bin/sh "/home/yhl/pira/extern/src/extrap/extrap-3.0/build/libtool" --silent --tag CXX --mode=relink g++ -g -O2 -o libextrap_python.la -rpath /home/yhl/pira/resources/../extern/install/extrap/lib libextrap_python_la-EXTRAP_Interface_wrap.lo libextrap.la ) libtool: install: /usr/bin/install -c .libs/libextrap_python.so.0.0.0T /home/yhl/pira/resources/../extern/install/extrap/lib/libextrap_python.so.0.0.0 libtool: install: (cd /home/yhl/pira/resources/../extern/install/extrap/lib && { ln -s -f libextrap_python.so.0.0.0 libextrap_python.so.0 || { rm -f libextrap_python.so.0 && ln -s libextrap_python.so.0.0.0 libextrap_python.so.0; }; }) libtool: install: (cd /home/yhl/pira/resources/../extern/install/extrap/lib && { ln -s -f libextrap_python.so.0.0.0 libextrap_python.so || { rm -f libextrap_python.so && ln -s libextrap_python.so.0.0.0 libextrap_python.so; }; }) libtool: install: /usr/bin/install -c .libs/libextrap_python.lai /home/yhl/pira/resources/../extern/install/extrap/lib/libextrap_python.la libtool: install: /usr/bin/install -c .libs/libextrap.a /home/yhl/pira/resources/../extern/install/extrap/lib/libextrap.a libtool: install: chmod 644 /home/yhl/pira/resources/../extern/install/extrap/lib/libextrap.a libtool: install: ranlib /home/yhl/pira/resources/../extern/install/extrap/lib/libextrap.a libtool: install: /usr/bin/install -c .libs/libextrap_python.a /home/yhl/pira/resources/../extern/install/extrap/lib/libextrap_python.a libtool: install: chmod 644 /home/yhl/pira/resources/../extern/install/extrap/lib/libextrap_python.a libtool: install: ranlib /home/yhl/pira/resources/../extern/install/extrap/lib/libextrap_python.a libtool: finish: PATH="/usr/local/automake/bin:/usr/local/mpich-4.1.2/bin:/opt/rh/llvm-toolset-10.0/root/usr/bin:/opt/rh/llvm-toolset-10.0/root/usr/sbin:/opt/rh/devtoolset-9/root/usr/bin:/usr/local/automake/bin:/usr/local/mpich-4.1.2/bin:/opt/rh/llvm-toolset-10.0/root/usr/bin:/opt/rh/llvm-toolset-10.0/root/usr/sbin:/usr/local/automake/bin:/opt/rh/llvm-toolset-10.0/root/usr/bin:/opt/rh/llvm-toolset-10.0/root/usr/sbin:/root/anaconda3/envs/py38/bin:/root/anaconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.392.b08-2.el7_9.x86_64/jre:/usr/local/hadoop-2.7.1/bin:/usr/local/hadoop-2.7.1/sbin:/root/bin:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.392.b08-2.el7_9.x86_64/jre:/usr/local/hadoop-2.7.1/bin:/usr/local/hadoop-2.7.1/sbin:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.392.b08-2.el7_9.x86_64/jre:/usr/local/hadoop-2.7.1/bin:/usr/local/hadoop-2.7.1/sbin:/sbin" ldconfig -n /home/yhl/pira/resources/../extern/install/extrap/lib

Libraries have been installed in: /home/yhl/pira/resources/../extern/install/extrap/lib

If you ever happen to want to link against installed libraries in a given directory, LIBDIR, you must either use libtool, and specify the full pathname of the library, or use the '-LLIBDIR' flag during linking and do at least one of the following:

• add LIBDIR to the 'LD_LIBRARY_PATH' environment variable during execution
• add LIBDIR to the 'LD_RUN_PATH' environment variable during linking
• use the '-Wl,-rpath -Wl,LIBDIR' linker flag
• have your system administrator add LIBDIR to '/etc/ld.so.conf'

See any operating system documentation about shared libraries for more information, such as the ld(1) and ld.so(8) manual pages.

mkdir -p /home/yhl/pira/resources/../extern/install/extrap/lib/extrap install ../src/python_wrapper/python/EXTRAP.py /home/yhl/pira/resources/../extern/install/extrap/lib install ../src/gui/AllFunctionsAsOneSurfacePlotWidget.py ../src/gui/AllFunctionsAsDifferentSurfacePlotWidget.py ../src/gui/ColorWidget.py ../src/gui/CubeFileReader.py ../src/gui/DataDisplay.py ../src/gui/DominatingFunctionsAsHeatMapWidget.py ../src/gui/DominatingFunctionsAsSingleScatterPlotWidget.py ../src/gui/GraphWidget.py ../src/gui/HeatMapGraphWidget.py ../src/gui/InterpolatedContourDisplayWidget.py ../src/gui/IsolinesDisplayWidget.py ../src/gui/MainWidget.py ../src/gui/MaxZAsSingleSurfacePlotWidget.py ../src/gui/MeasurementPointsPlotWidget.py ../src/gui/ModelerWidget.py ../src/gui/MultiParameterSimpleModeler.py ../src/gui/ParameterValueSlider.py ../src/gui/PlotTypeSelector.py ../src/gui/SelectorWidget.py ../src/gui/SingleParameterSimpleModeler.py ../src/gui/SingleParameterRefiningModeler.py ../src/gui/SingleParameterExhaustiveModeler.py ../src/gui/TreeModel.py ../src/gui/TreeView.py ../src/gui/TwoParameterCubeFileReader.py ../src/gui/Utils.py ../src/tools/print/extrap-print.py ../src/tools/modeler/extrap-modeler.py ../src/tools/docopt/docopt.py /home/yhl/pira/resources/../extern/install/extrap/lib/extrap mkdir -p /home/yhl/pira/resources/../extern/install/extrap/bin for e in print modeler; do
echo "#!/root/anaconda3/envs/py38/bin/python" > /home/yhl/pira/resources/../extern/install/extrap/bin/extrap-$e;
sed 's|@LIBDIR@|/home/yhl/pira/resources/../extern/install/extrap/lib|g' ../src/tools/$e/extrap-$e.py.in >> /home/yhl/pira/resources/../extern/install/extrap/bin/extrap-$e;
chmod a+x /home/yhl/pira/resources/../extern/install/extrap/bin/extrap-$e;
done mkdir -p /home/yhl/pira/resources/../extern/install/extrap/bin echo "#!/root/anaconda3/envs/py38/bin/python" > /home/yhl/pira/resources/../extern/install/extrap/bin/extrap sed 's|@LIBDIR@|/home/yhl/pira/resources/../extern/install/extrap/lib|g' ../src/gui/gui.py.in >> /home/yhl/pira/resources/../extern/install/extrap/bin/extrap chmod a+x /home/yhl/pira/resources/../extern/install/extrap/bin/extrap make install-exec-hook make[2]: Entering directory /home/yhl/pira/extern/src/extrap/extrap-3.0/build' if [ -a /home/yhl/pira/resources/../extern/install/extrap/lib/libextrap_python.dylib ]; then cp /home/yhl/pira/resources/../extern/install/extrap/lib/libextrap_python.dylib /home/yhl/pira/resources/../extern/install/extrap/lib/libextrap_python.so; fi make[2]: Leaving directory /home/yhl/pira/extern/src/extrap/extrap-3.0/build' /usr/bin/mkdir -p '/home/yhl/pira/resources/../extern/install/extrap/share' /usr/bin/install -c -m 644 ../share/tvalues.val ../share/input.txt '/home/yhl/pira/resources/../extern/install/extrap/share' make[1]: Leaving directory `/home/yhl/pira/extern/src/extrap/extrap-3.0/build'

[jplehr]

Thanks for the details. I'm not sure I'm fully following along though.

The error that you shared above was about missing includes in the MetaCG project, not about a failing link step. Did you resolve the include issue and then came across a link failure? Or did the Extra-P build fail due to some issue?

I do wonder why we do not see such issues on our end in the CI, independent of whether we use an Extra-P installation from our module system or with a freshly built version that uses the same script.

[Masterwater-y]

The previous error was my bad. I only deleted metacg/build before rerunning ./build_submodule each time, which caused $extrap_install_dir to not be set correctly, resulting in the extrap-related dependencies not being found. Now I have successfully built metacg and completed the remaining steps. However, when I ran run.sh for GameOfLife, I encountered a new error as shown below. By the way, it is also on the host system, CentOS7 Do I need to try building it again on the host system of Ubuntu? Are there significant differences between Docker and the host system? I have tried deploying with Docker, and encountered an error at the GameOfLife's run.sh step (as mentioned in another issue https://github.com/tudasc/PIRA/issues/8 ).

----- Build GameOfLife / build call graph ----- SerialGoL.cpp: In member function ‘const int GameOfLife::getNumLiveNeighbors(int, int) const’: SerialGoL.cpp:94:45: warning: comparison between signed and unsigned integer expressions [-Wsign-compare] if(neighbor >= 0 && neighbor < gridA.size()){ ^ [Warning] Unable to determine direct callee. [Warning] Unable to determine direct callee. ----- Running Pira ----- Using /root/.local/share/pira for runtime files [INFO in '/home/yhl/pira/lib/Pira.py' in 'main' line 168]: Running PIRA in compiletime filtering with configuration /home/yhl/pira/test/integration/GameOfLife/gol_config.json [WARNING in '/home/yhl/pira/lib/Pira.py' in 'process_args_for_extrap' line 147]: At least 5 repetitions are recommended for Extra-P modelling. [INFO in '/home/yhl/pira/lib/Checker.py' in 'check_configfile_v2' line 84]: Creating Cubes-Directory /root/.local/share/pira/gol_cubes [INFO in '/home/yhl/pira/lib/Pira.py' in 'main' line 190]: Running the local case [INFO in '/home/yhl/pira/lib/Pira.py' in 'execute_with_config' line 40]: Building vanilla version for baseline measurements [INFO in '/home/yhl/pira/lib/Pira.py' in 'execute_with_config' line 46]: Running baseline measurements [INFO in '/home/yhl/pira/lib/Pira.py' in 'execute_with_config' line 57]: Running instrumentation iteration 0 [ERROR in '/home/yhl/pira/lib/Utility.py' in 'shell' line 346]: Utility::shell: Caught Exception Command 'pgis_pira --scorep-out --static /home/yhl/pira/test/integration/GameOfLife/../../../extern/install/metacg/bin//gol_ct.mcg' returned non-zero exit status 127. [ERROR in '/home/yhl/pira/lib/Analyzer.py' in 'analyze_local' line 111]: Utility::shell: Running command pgis_pira --scorep-out --static /home/yhl/pira/test/integration/GameOfLife/../../../extern/install/metacg/bin//gol_ct.mcg did not succeed [ERROR in '/home/yhl/pira/lib/Pira.py' in 'execute_with_config' line 111]: Pira::execute_with_config: Problem during preparation of run. Message: Problem in Analyzer [ERROR in '/home/yhl/pira/lib/Pira.py' in 'main' line 256]: Runner.run caught exception. Message: Problem in Analyzer Error opening instrumentation file for iteration 0: [Errno 2] No such file or directory: '../../../extern/install/metacg/bin/out/instrumented-gol_ct_it-0.txt' Error opening instrumentation file for iteration 1: [Errno 2] No such file or directory: '../../../extern/install/metacg/bin/out/instrumented-gol_ct_it-1.txt' Error opening instrumentation file for iteration 2: [Errno 2] No such file or directory: '../../../extern/install/metacg/bin/out/instrumented-gol_ct_it-2.txt'

[jplehr]

Thanks for clarifying.

No, there is no need to switch to a different OS. The error you are seeing is due to the CLI of MetaCG change from v0.4.0 (previously tracked by PIRA) and v0.6.x (that I think is now checked out to work around the build issue you reported). We have a pending patch in preparation to move PIRA forward to v0.6.1 and adjust all CLI options, but do see another test failure that we are working on. That will likely need another patch to MetaCG before we can lift PIRA to v0.6.x.

Unfortunately, some assumptions in PIRA and latest MetaCG had a bit diverged and we currently fix that.

[Masterwater-y]

Thank you very much，waiting for the new release sincerely

[pearzt]

We have released PIRA v0.6.0 which should fix the reported issue. Mostly, it updates the dependency to MetaCG to version 0.6.2 and adjusts the PIRA code accordingly.

@Masterwater-y We would be interested to hear whether the new release works for you so that we then could close this issue. Please test with a clean checkout so that there are no leftovers from older versions laying around.

[jplehr]

I'll close this for now, due to inactivity.