Get M2 to work with CLion (problem is mostly build usage of OpenMP)

[mikestillman]

At some point, Macaulay2 stopped being able to be used by the CLion IDE. It loads fine, but the code analysis and refactoring tools mostly don't work currently. I think clangd is getting confused by command line options arising from our usage of -Xclang and similar (see below).

I would like to get this functional again. It appears to be basically related to our handling of OpenMP.

If I make a compilation database (via cmake, using -DCMAKE_EXPORT_COMPILE_COMMANDS=on), and symbolically link the resulting file compile_commands.json into the M2 directory, I have the same problems as with opening the cmake project: CLion can't find system headers, or headers that although they are findable using compile options, it seems to fail at that. However, if I remove all uses (in the compile_commands.json file, the following absolute link is what I see on my machine) of -Xpreprocessor -fopenmp /opt/homebrew/opt/libomp/lib/libomp.dylib and -Xclang, and open the project, then code analysis, finding header files, and refactoring, all work. Of course, building by cmake is not supported with this type of project.

The problem (as partially noticed in #2883 for instance) appears to be that using (at least with an Apple M1, running MacOS) apple clang with openmp has the following issues:

• ffpack places -Xpreprocessor -fopenmp into our cmake variables.
• OpenMP cmake routines add in -Xclang -fopenmp, creating a conflict (one of the -fopenmp's gets removed from the command line during some deduplication step, causing clangd to get confused, I believe).
• Actually, with llvm clang++ from brew, we do not need -Xclang or -Xpreprocessor, but both ffpack and cmake appear to put them in, in any case.
• the actual file name for the libomp dylib is on the command line /opt/homebrew/opt/libomp/lib/libomp.dylib. This appears to confuse clangd in CLion as well (for some reason it tries to read it as a text file). (Perhaps it should be -L/opt/homebrew/opt/libomp/lib -lomp? I haven't tried that yet to see if clangd gets less confused)

How should we fix this? I'm not sure what the best solution is, hence this issue. Some possibles:

• replace, after setting up ffpack, -Xpreprocessor with -Xclang. This might mean we don't need to remove the -Xclang later. We still need to fix the libomp line, I'm not sure where this is being included in yet (ffpack or the cmake find_package(OpenMP)).
• somehow know if we are using apple clang, vs homebrew clang (maybe we already know this, and the code is just choosing the wrong thing?). If homebrew version, use essentially the linux solution (simply -fopenmp -lomp?, although we need to tell cmake where omp library is)

Other notes:

• CLion cannot find all header files until after the first build of M2 has been done, because of the generated include files.
• it appears not possible to compile without OpenMP, if it is on the system? e.g. ffpack always uses it? I haven't experimented to see if this is really the case yet.

[mahrud]

Try applying this patch:

diff --git a/M2/Macaulay2/e/CMakeLists.txt b/M2/Macaulay2/e/CMakeLists.txt
index ce702082fe..bd23b68304 100644
--- a/M2/Macaulay2/e/CMakeLists.txt
+++ b/M2/Macaulay2/e/CMakeLists.txt
@@ -359,10 +359,6 @@ if(EIGEN3_FOUND)
   target_link_libraries(M2-engine PUBLIC Eigen3::Eigen)
 endif()

-if(OpenMP_FOUND)
-  target_link_libraries(M2-engine PUBLIC OpenMP::OpenMP_CXX)
-endif()
-
 # Compiler warning flags
 target_compile_options(M2-engine PRIVATE
   -Wno-cast-qual # FIXME
-- 
2.40.1

I'd rather not actually commit it to git yet, because this issue will be resolved with the next version of cmake, but it should resolve the clion issue. Note that M2 will still build with OpenMP because ffpack includes the link flags.

Also, I don't think #2883 is related to this.

[mikestillman]

This change allows me to use cmake 3.26.4 to compile M2, which is good, thanks!

It doesn't quite allow CLion to work yet. The problem appears to be that ffpack (or someone?) is placing on the compile command line the file /opt/homebrew/opt/libomp/lib/libomp.dylib directly. This appears to be confusing clangd. I get megabytes of errors as it tries to parse this as a c++ file, I believe.

If I remove all uses of this file from the compile_commands.json file, then CLion loads the project with no errors.

Is there any way we can replace this part of the command line with something like -L /opt/homebrew/opt/libomp/lib -lompp ?

[mahrud]

Can you paste one compile command? (before modification)

[mikestillman]

Here are 2 entries (there is one for each file).

{
  "directory": "/Users/mike/src/M2-current-branches/M2-development/M2/BUILD/mike/builds.tmp/cmake-appleclang",
  "command": "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DNDEBUG -DOM_NDEBUG -DSING_NDEBUG -I/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/e -I/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/d -I/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/c -I/Users/mike/src/M2-current-branches/M2-development/M2/BUILD/mike/builds.tmp/cmake-appleclang/Macaulay2/d -I/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/e/mathic -I/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/e/memtailor -I/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/e/mathicgb -I/opt/homebrew/opt/readline/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/usr/include/libxml2 -I/opt/homebrew/opt/gdbm/include -I/opt/homebrew/opt/libatomic_ops/include -isystem /opt/homebrew/include/eigen3 -isystem /opt/homebrew/include -isystem /opt/homebrew/Cellar/fflas-ffpack/2.5.0/include -isystem /opt/homebrew/opt/libomp/include -isystem /opt/homebrew/Cellar/givaro/4.2.0/include -isystem /opt/homebrew/opt/gmp/include -O2 -g -DNDEBUG -std=gnu++17 -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk -mmacosx-version-min=12.6 -Wuninitialized -g --sysroot=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk -I/Users/mike/src/M2-current-branches/M2-development/M2/BUILD/mike/builds.tmp/cmake-appleclang/usr-host/include -I/Users/mike/src/M2-current-branches/M2-development/M2/include -I/Users/mike/src/M2-current-branches/M2-development/M2/BUILD/mike/builds.tmp/cmake-appleclang/include -Wall -Wextra -Wfatal-errors -Wcast-qual -Wno-unused-parameter -Wno-attributes -Wno-unused-command-line-argument -Wno-cast-qual -Wno-sign-compare -Wno-unused-local-typedefs -Wno-deprecated-register -Wno-mismatched-tags -Wno-unused-variable -Xpreprocessor -fopenmp /opt/homebrew/opt/libomp/lib/libomp.dylib -o Macaulay2/e/CMakeFiles/M2-engine.dir/matrix-kbasis.cpp.o -c /Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/e/matrix-kbasis.cpp",
  "file": "/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/e/matrix-kbasis.cpp",
  "output": "Macaulay2/e/CMakeFiles/M2-engine.dir/matrix-kbasis.cpp.o"
},
{
  "directory": "/Users/mike/src/M2-current-branches/M2-development/M2/BUILD/mike/builds.tmp/cmake-appleclang",
  "command": "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DNDEBUG -DOM_NDEBUG -DSING_NDEBUG -I/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/e -I/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/d -I/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/c -I/Users/mike/src/M2-current-branches/M2-development/M2/BUILD/mike/builds.tmp/cmake-appleclang/Macaulay2/d -I/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/e/mathic -I/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/e/memtailor -I/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/e/mathicgb -I/opt/homebrew/opt/readline/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/usr/include/libxml2 -I/opt/homebrew/opt/gdbm/include -I/opt/homebrew/opt/libatomic_ops/include -isystem /opt/homebrew/include/eigen3 -isystem /opt/homebrew/include -isystem /opt/homebrew/Cellar/fflas-ffpack/2.5.0/include -isystem /opt/homebrew/opt/libomp/include -isystem /opt/homebrew/Cellar/givaro/4.2.0/include -isystem /opt/homebrew/opt/gmp/include -O2 -g -DNDEBUG -std=gnu++17 -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk -mmacosx-version-min=12.6 -Wuninitialized -g --sysroot=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk -I/Users/mike/src/M2-current-branches/M2-development/M2/BUILD/mike/builds.tmp/cmake-appleclang/usr-host/include -I/Users/mike/src/M2-current-branches/M2-development/M2/include -I/Users/mike/src/M2-current-branches/M2-development/M2/BUILD/mike/builds.tmp/cmake-appleclang/include -Wall -Wextra -Wfatal-errors -Wcast-qual -Wno-unused-parameter -Wno-attributes -Wno-unused-command-line-argument -Wno-cast-qual -Wno-sign-compare -Wno-unused-local-typedefs -Wno-deprecated-register -Wno-mismatched-tags -Wno-unused-variable -Xpreprocessor -fopenmp /opt/homebrew/opt/libomp/lib/libomp.dylib -o Macaulay2/e/CMakeFiles/M2-engine.dir/matrix-sort.cpp.o -c /Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/e/matrix-sort.cpp",
  "file": "/Users/mike/src/M2-current-branches/M2-development/M2/Macaulay2/e/matrix-sort.cpp",
  "output": "Macaulay2/e/CMakeFiles/M2-engine.dir/matrix-sort.cpp.o"
},

[mahrud]

Do you know what is inserting /opt/homebrew/opt/libomp/lib/libomp.dylib? What's the content of $(brew --prefix fflas-ffpack)/lib/pkgconfig/fflas-ffpack.pc on your machine?

[mikestillman]

Aahh, can I just edit this file?

prefix=/opt/homebrew/Cellar/fflas-ffpack/2.5.0
exec_prefix=/opt/homebrew/Cellar/fflas-ffpack/2.5.0
libdir=/opt/homebrew/Cellar/fflas-ffpack/2.5.0/lib
includedir=/opt/homebrew/Cellar/fflas-ffpack/2.5.0/include

Name: fflas-ffpack
Description: Finite Field Linear Algebra Suroutines/Package
URL: http://github.com/linbox-team/fflas-ffpack
Version: 2.5.0
Requires: givaro >= 4.1.2
Libs: -Xpreprocessor -fopenmp -I/opt/homebrew/opt/libomp/include /opt/homebrew/opt/libomp/lib/libomp.dylib  
Cflags: -I${prefix}/include    -Xpreprocessor -fopenmp -I/opt/homebrew/opt/libomp/include /opt/homebrew/opt/libomp/lib/libomp.dylib  

[mahrud]

You can, though it might be simpler if I just edit this file in the fflas-ffpack bottles.

Including a dynamic library on the compile line seems like normal syntax to me though. Does CLion show you the exact compile command and error output?

[mikestillman]

It does seem like a fine way to include a library. CLion does not show me what is going on, I need to piece it together.

I will still try doing it via -L, -lomp. If that works, I might want to use that (either locally on my machine, or having you change it in the bottle). I'm not sure in that case which program is at fault (CLion, clangd, clang++, maybe some other program CLion is running without my knowledge?).

[mahrud]

How do you know it's related to openmp?

[mikestillman]

After using your fix to e/CMakeLists.txt, using the compile_commands.json file as is causes CLion to report many errors (which keeps it from doing code analysis). However, if I remove the libomp.dylib in all entries, and use that new file instead, it reports no errors, and can do code analysis fine (and finds all header files etc). I suppose it could be something else, but that file for some reason seems to be screwing things up. Now, I'm not sure which cached cmake variable it is using to place this file name in. There are a number of ffpack cmake cached variables that have this name in it. I found it hard (actually, impossible so far) to remove this libomp.dylib file from the compilation data base (by changing ffpack things, or M2 things. I can fix it by editing the .json file, but I want to avoid that, so that hopefully I can use cmake building inside CLion)

[mahrud]

I think if you remove the dylib from $(brew --prefix fflas-ffpack)/lib/pkgconfig/fflas-ffpack.pc then run:

cmake -U*FFPACK* -U*pkgconfig* . # <-- see the dot

then you wouldn't need to edit the json file.

[mikestillman]

Thanks @mahrud, I finally have CLion working properly, with a cmake build of M2. @moorewf Could you please try this?

Here are some steps I needed to make. In the end, I think I don't need to do the last step (modify e/CMakeLists.txt).

Change to fflas-ffpack.pc file

My ffpack.pc file is now the following. I modified the Libs line to not explicitly mention the exact dylib libomp file. I also changed the -Xpreprocessor to -Xclang.

bash-3.2$ cat $(brew --prefix fflas-ffpack)/lib/pkgconfig/fflas-ffpack.pc
prefix=/opt/homebrew/Cellar/fflas-ffpack/2.5.0
exec_prefix=/opt/homebrew/Cellar/fflas-ffpack/2.5.0
libdir=/opt/homebrew/Cellar/fflas-ffpack/2.5.0/lib
includedir=/opt/homebrew/Cellar/fflas-ffpack/2.5.0/include

Name: fflas-ffpack
Description: Finite Field Linear Algebra Suroutines/Package
URL: http://github.com/linbox-team/fflas-ffpack
Version: 2.5.0
Requires: givaro >= 4.1.2
Libs: -Xclang -fopenmp -I/opt/homebrew/opt/libomp/include -L/opt/homebrew/opt/libomp/lib -lomp
Cflags: -I${prefix}/include    -Xclang -fopenmp -I/opt/homebrew/opt/libomp/include

Changes to make in CLion

In CLion, I gave the following options to cmake (in settings), for all 4 build types:

-DCMAKE_PREFIX_PATH=/opt/homebrew/opt/libffi -DBUILD_NATIVE=off -DBUILD_TESTING=on -DBUILD_DOCS=on -DCMAKE_EXPORT_COMPILE_COMMANDS=on

The first 2 options are definitely needed on Apple M1, the others were my choice.

Not needed, I think, but at first I was doing this step

In my Macaulay2/e/CMakeLists.txt file, I made the following diff. I believe I don't need to do this step.

diff --git a/M2/Macaulay2/e/CMakeLists.txt b/M2/Macaulay2/e/CMakeLists.txt
index ce702082f..b3e7ee6e6 100644
--- a/M2/Macaulay2/e/CMakeLists.txt
+++ b/M2/Macaulay2/e/CMakeLists.txt
@@ -359,9 +359,9 @@ if(EIGEN3_FOUND)
   target_link_libraries(M2-engine PUBLIC Eigen3::Eigen)
 endif()
 
-if(OpenMP_FOUND)
-  target_link_libraries(M2-engine PUBLIC OpenMP::OpenMP_CXX)
-endif()
+# if(OpenMP_FOUND)
+#   target_link_libraries(M2-engine PUBLIC OpenMP::OpenMP_CXX)
+# endif()
 
 # Compiler warning flags
 target_compile_options(M2-engine PRIVATE

[mahrud]

@mikestillman CLion working now?

[mahrud]

Closing this, feel free to reopen if there are more CLion issues.