c3c icon indicating copy to clipboard operation
c3c copied to clipboard
verified publisher icon c3lang

CMake fails to catch LLD library location in a non-FHS layout

Open Ashvith10 opened this issue 9 months ago • 1 comments

For more context, I am packaging version 0.5.1, all the way to the latest tag.

It seems that LLD library directory isn't captured in the variable ${LLVM_LIBRARY_DIRS} for Guix (and Nix) - perhaps, it could be due to how it is packaged, as opposed to FHS-compliant distros, or maybe because CMakeLists.txt is missing something. Both llvm and lld are two distinct packages in Guix (and Nix):

ashvith@wisteria ~/Desktop/c3c$ tree $(guix build llvm@18) -L 1
/gnu/store/g0y4x0cf5w9yvj9xybpcb0lryf6aym31-llvm-18.1.8-opt-viewer
└── share
/gnu/store/xq9698hniwgjdmwknain5myfdr6p70qb-llvm-18.1.8
├── bin
├── etc
├── include
├── lib
└── share

8 directories, 0 files
ashvith@wisteria ~/Desktop/c3c$ tree $(guix build lld@18) -L 2
/gnu/store/qmciqz70f3ps3qxb7nnpj9xrwizasfxa-lld-18.1.8
├── bin
│   ├── ld64.lld
│   ├── ld.lld
│   ├── lld
│   ├── lld-link
│   └── wasm-ld
├── etc
│   └── ld.so.cache
├── include
│   └── lld
├── lib
│   ├── cmake
│   ├── liblldCOFF.a
│   ├── liblldCommon.a
│   ├── liblldELF.a
│   ├── liblldMachO.a
│   ├── liblldMinGW.a
│   └── liblldWasm.a
└── share
    └── doc

9 directories, 12 files

So far, I have experimented with:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 5010307..835214b 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -121,14 +121,22 @@ if(CMAKE_C_COMPILER_ID STREQUAL "MSVC")
 else()
     if (NOT C3_LLVM_VERSION STREQUAL "auto")
         find_package(LLVM ${C3_LLVM_VERSION} REQUIRED CONFIG)
+        find_package(LLD ${C3_LLVM_VERSION} REQUIRED CONFIG)
     else()
         find_package(LLVM REQUIRED CONFIG)
+        find_package(LLD REQUIRED CONFIG)
     endif()
 endif()
 
+set(LLD_LIBRARY_DIRS "${LLD_INSTALL_PREFIX}/lib")
+
 message(STATUS "Found LLVM ${LLVM_PACKAGE_VERSION}")
 message(STATUS "Using LLVMConfig.cmake in: ${LLVM_DIR}")
-message(STATUS "Libraries located in: ${LLVM_LIBRARY_DIRS}")
+message(STATUS "LLVM Libraries located in: ${LLVM_LIBRARY_DIRS}")
+
+message(STATUS "Found LLD ${LLD_PACKAGE_VERSION}")
+message(STATUS "Using LLDConfig.cmake in: ${LLD_DIR}")
+message(STATUS "LLD Libraries located in: ${LLD_LIBRARY_DIRS}")
 
 if(LLVM_ENABLE_RTTI)
     message(STATUS "LLVM was built with RTTI")
@@ -140,6 +148,10 @@ include_directories(${LLVM_INCLUDE_DIRS})
 link_directories(${LLVM_LIBRARY_DIRS})
 add_definitions(${LLVM_DEFINITIONS})
 
+include_directories(${LLD_INCLUDE_DIRS})
+link_directories(${LLD_LIBRARY_DIRS})
+add_definitions(${LLD_DEFINITIONS})
+
 if(NOT C3_LINK_DYNAMIC)
     set(LLVM_LINK_COMPONENTS
 	    AllTargetsAsmParsers
@@ -175,31 +187,31 @@ if(NOT C3_LINK_DYNAMIC)
 
     # These don't seem to be reliable on windows.
     message(STATUS "using find_library")
-    find_library(LLD_COFF NAMES lldCOFF.lib lldCOFF.a liblldCOFF.a PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    find_library(LLD_COMMON NAMES lldCommon.lib lldCommon.a liblldCommon.a PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    find_library(LLD_ELF NAMES lldELF.lib lldELF.a liblldELF.a PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    find_library(LLD_MACHO NAMES lldMachO.lib lldMachO.a liblldMachO.a PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    find_library(LLD_MINGW NAMES lldMinGW.lib lldMinGW.a liblldMinGW.a PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    find_library(LLD_WASM NAMES lldWasm.lib lldWasm.a liblldWasm.a PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_COFF NAMES lldCOFF.lib lldCOFF.a liblldCOFF.a PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_COMMON NAMES lldCommon.lib lldCommon.a liblldCommon.a PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_ELF NAMES lldELF.lib lldELF.a liblldELF.a PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_MACHO NAMES lldMachO.lib lldMachO.a liblldMachO.a PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_MINGW NAMES lldMinGW.lib lldMinGW.a liblldMinGW.a PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_WASM NAMES lldWasm.lib lldWasm.a liblldWasm.a PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
 else()
-    find_library(LLVM NAMES libLLVM.so PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    set(llvm_libs ${LLVM})
+    find_library(LLVM NAMES libLLVM.so PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    set(llvm_libs ${LLVM} ${LLD})
 
     # These don't seem to be reliable on windows.
     message(STATUS "using find_library")
-    find_library(LLD_COFF NAMES liblldCOFF.so PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    find_library(LLD_COMMON NAMES liblldCommon.so PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    find_library(LLD_ELF NAMES liblldELF.so PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    find_library(LLD_MACHO NAMES liblldMachO.so PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    find_library(LLD_MINGW NAMES liblldMinGW.so PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    find_library(LLD_WASM NAMES liblldWasm.so PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_COFF NAMES liblldCOFF.so PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_COMMON NAMES liblldCommon.so PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_ELF NAMES liblldELF.so PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_MACHO NAMES liblldMachO.so PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_MINGW NAMES liblldMinGW.so PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_WASM NAMES liblldWasm.so PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
 endif()
 
 file(REMOVE_RECURSE ${CMAKE_BINARY_DIR}/lib)
 file(COPY ${CMAKE_SOURCE_DIR}/lib DESTINATION ${CMAKE_BINARY_DIR})
 
 if (${LLVM_PACKAGE_VERSION} VERSION_GREATER_EQUAL 16)
-    find_library(LLD_LOONG NAMES libLLVMLoongArchCodeGen.lib libLLVMLoongArchAsmParser.lib libLLVMLoongArchCodeGen.a libLLVMLoongArchAsmParser.a PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_LOONG NAMES libLLVMLoongArchCodeGen.lib libLLVMLoongArchAsmParser.lib libLLVMLoongArchCodeGen.a libLLVMLoongArchAsmParser.a PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
     set(lld_libs
         ${LLD_COFF}
         ${LLD_COMMON}

and it seems to be building the way it should be. For the older versions, I can manage this with a patch. However, for the newest release, I would really appreciate some help in making the above changes compatible so that it works without any issues on other platforms.

Ashvith10 avatar Mar 15 '25 08:03 Ashvith10

I have the same problem with homebrew installing LLVM and LLD in different places, since that seems contradict what LLVM expects.

lerno avatar Mar 15 '25 19:03 lerno

@lerno Were you able to resolve this? If not, could you look at this patch? (more context below)

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 5010307..835214b 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -121,14 +121,22 @@ if(CMAKE_C_COMPILER_ID STREQUAL "MSVC")
 else()
     if (NOT C3_LLVM_VERSION STREQUAL "auto")
         find_package(LLVM ${C3_LLVM_VERSION} REQUIRED CONFIG)
+        find_package(LLD ${C3_LLVM_VERSION} REQUIRED CONFIG)
     else()
         find_package(LLVM REQUIRED CONFIG)
+        find_package(LLD REQUIRED CONFIG)
     endif()
 endif()
 
+set(LLD_LIBRARY_DIRS "${LLD_INSTALL_PREFIX}/lib")
+
 message(STATUS "Found LLVM ${LLVM_PACKAGE_VERSION}")
 message(STATUS "Using LLVMConfig.cmake in: ${LLVM_DIR}")
-message(STATUS "Libraries located in: ${LLVM_LIBRARY_DIRS}")
+message(STATUS "LLVM Libraries located in: ${LLVM_LIBRARY_DIRS}")
+
+message(STATUS "Found LLD ${LLD_PACKAGE_VERSION}")
+message(STATUS "Using LLDConfig.cmake in: ${LLD_DIR}")
+message(STATUS "LLD Libraries located in: ${LLD_LIBRARY_DIRS}")
 
 if(LLVM_ENABLE_RTTI)
     message(STATUS "LLVM was built with RTTI")
@@ -140,6 +148,10 @@ include_directories(${LLVM_INCLUDE_DIRS})
 link_directories(${LLVM_LIBRARY_DIRS})
 add_definitions(${LLVM_DEFINITIONS})
 
+include_directories(${LLD_INCLUDE_DIRS})
+link_directories(${LLD_LIBRARY_DIRS})
+add_definitions(${LLD_DEFINITIONS})
+
 if(NOT C3_LINK_DYNAMIC)
     set(LLVM_LINK_COMPONENTS
 	    AllTargetsAsmParsers
@@ -175,31 +187,31 @@ if(NOT C3_LINK_DYNAMIC)
 
     # These don't seem to be reliable on windows.
     message(STATUS "using find_library")
-    find_library(LLD_COFF NAMES lldCOFF.lib lldCOFF.a liblldCOFF.a PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    find_library(LLD_COMMON NAMES lldCommon.lib lldCommon.a liblldCommon.a PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    find_library(LLD_ELF NAMES lldELF.lib lldELF.a liblldELF.a PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    find_library(LLD_MACHO NAMES lldMachO.lib lldMachO.a liblldMachO.a PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    find_library(LLD_MINGW NAMES lldMinGW.lib lldMinGW.a liblldMinGW.a PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    find_library(LLD_WASM NAMES lldWasm.lib lldWasm.a liblldWasm.a PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_COFF NAMES lldCOFF.lib lldCOFF.a liblldCOFF.a PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_COMMON NAMES lldCommon.lib lldCommon.a liblldCommon.a PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_ELF NAMES lldELF.lib lldELF.a liblldELF.a PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_MACHO NAMES lldMachO.lib lldMachO.a liblldMachO.a PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_MINGW NAMES lldMinGW.lib lldMinGW.a liblldMinGW.a PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_WASM NAMES lldWasm.lib lldWasm.a liblldWasm.a PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
 else()
-    find_library(LLVM NAMES libLLVM.so PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    set(llvm_libs ${LLVM})
+    find_library(LLVM NAMES libLLVM.so PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    set(llvm_libs ${LLVM} ${LLD})
 
     # These don't seem to be reliable on windows.
     message(STATUS "using find_library")
-    find_library(LLD_COFF NAMES liblldCOFF.so PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    find_library(LLD_COMMON NAMES liblldCommon.so PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    find_library(LLD_ELF NAMES liblldELF.so PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    find_library(LLD_MACHO NAMES liblldMachO.so PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    find_library(LLD_MINGW NAMES liblldMinGW.so PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
-    find_library(LLD_WASM NAMES liblldWasm.so PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_COFF NAMES liblldCOFF.so PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_COMMON NAMES liblldCommon.so PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_ELF NAMES liblldELF.so PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_MACHO NAMES liblldMachO.so PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_MINGW NAMES liblldMinGW.so PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_WASM NAMES liblldWasm.so PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
 endif()
 
 file(REMOVE_RECURSE ${CMAKE_BINARY_DIR}/lib)
 file(COPY ${CMAKE_SOURCE_DIR}/lib DESTINATION ${CMAKE_BINARY_DIR})
 
 if (${LLVM_PACKAGE_VERSION} VERSION_GREATER_EQUAL 16)
-    find_library(LLD_LOONG NAMES libLLVMLoongArchCodeGen.lib libLLVMLoongArchAsmParser.lib libLLVMLoongArchCodeGen.a libLLVMLoongArchAsmParser.a PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH)
+    find_library(LLD_LOONG NAMES libLLVMLoongArchCodeGen.lib libLLVMLoongArchAsmParser.lib libLLVMLoongArchCodeGen.a libLLVMLoongArchAsmParser.a PATHS ${LLVM_LIBRARY_DIRS} ${LLD_LIBRARY_DIRS} NO_DEFAULT_PATH)
     set(lld_libs
         ${LLD_COFF}
         ${LLD_COMMON}

I am packaging c3c, all the way from 0.5.1 to whatever is the present version - in this particular case, patch is targeted for 0.5.1. It also includes a patch to remove miniz and use zlib instead (not relevant to the conversation, but to just let you know, if in case the patch does not apply). Maybe, instead of setting a new variable LLD_LIBRARY_DIRS, I should probably be making use of LLD_CMAKE_DIR.

Ashvith10 avatar Aug 07 '25 16:08 Ashvith10

If you make a PR for it I can test it on CI and locally?

lerno avatar Aug 26 '25 14:08 lerno

@lerno would it be okay if I share with you the changes targetting c3 v0.5.1? I am using the oldest version so that in the possible future that a bootstrap is required, it would be really helpful to package in Guix.

Ashvith10 avatar Aug 26 '25 18:08 Ashvith10

@lerno can you check the PR at #2436?

Ashvith10 avatar Aug 26 '25 18:08 Ashvith10

I have removed the PR. @lerno , would it be possible to fetch my changes to your local repository, so that you can test it on the platforms you're having issues with?

  1. You can add my fork using git remote add c3c-ashvith https://github.com/Ashvith10/c3c
  2. Next, fetch the branches using: git fetch c3c-ashvith
  3. Then, switch to the branch with: git checkout -b fix-issues-with-missing-lld-directory c3c-ashvith/fix-issues-with-missing-lld-directory

As I have mentioned earlier, since I am working with bootstrappability in mind, I am targetting the older version, but I believe that these changes should work with newer versions too.

Ashvith10 avatar Aug 26 '25 20:08 Ashvith10

I noted that the latest C3 cmake script now works in these cases anyway. But you need this for other things, right?

lerno avatar Aug 27 '25 09:08 lerno

Unfortunately, I could not catch what you're trying to say. Are you saying that the latest version of c3c does not have this issue while building? I could try building it once and then share the output with you.

Ashvith10 avatar Aug 27 '25 14:08 Ashvith10

Please do

lerno avatar Sep 12 '25 18:09 lerno