Cmake build system implementation

[madjesc]

Summary of changes

This pull request includes following changes or fixes.

• Add the cmake build system mentioned in #521

Description

Added CMakeLists.txt for cmake to work and do some changes in code to not use config.h

Type of change

• New feature (non-breaking change which adds functionality)

Checklist:

• [x] My code follows the style guidelines of this project
• [x] I have performed a self-review of my own code
• [x] I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [x] My changes generate no new warnings
• [x] I have confirmed my fix is effective or that my feature works

Progress

• [x] add CMakeLists.txt to every part of the code
• [x] make it compile
• [ ] make it installs
• [x] make it generate the config files
• [x] make it generate doc files and man pages
• [x] do compile time tests against autotools

[madjesc]

Also, for some reason ICU cannot find the resource bundles even thought they are linked to the binary. I'm probably missing a compile flag, feature or something. Any idea on that?

[mcardenas@kasane build]$ ./src/ltfs
LTFS11286E Cannot load messages: failed to open resource bundle (2)
LTFS11293E Cannot load messages for libltfs (-1)
LTFS9011E Logging initialization failed
LTFS10000E (could not generate message)
LTFS11286E Cannot load messages: failed to open resource bundle (2)
LTFS10012E (could not generate message)

[piste-jp]

Also, for some reason ICU cannot find the resource bundles even thought they are linked to the binary. I'm probably missing a compile flag, feature or something. Any idea on that?

[mcardenas@kasane build]$ ./src/ltfs
LTFS11286E Cannot load messages: failed to open resource bundle (2)
LTFS11293E Cannot load messages for libltfs (-1)
LTFS9011E Logging initialization failed
LTFS10000E (could not generate message)
LTFS11286E Cannot load messages: failed to open resource bundle (2)
LTFS10012E (could not generate message)

Did you specify static link against resource bundle created by ICU?

[madjesc]

Also, for some reason ICU cannot find the resource bundles even thought they are linked to the binary. I'm probably missing a compile flag, feature or something. Any idea on that?

[mcardenas@kasane build]$ ./src/ltfs
LTFS11286E Cannot load messages: failed to open resource bundle (2)
LTFS11293E Cannot load messages for libltfs (-1)
LTFS9011E Logging initialization failed
LTFS10000E (could not generate message)
LTFS11286E Cannot load messages: failed to open resource bundle (2)
LTFS10012E (could not generate message)

Did you specify static link against resource bundle created by ICU?

It's dinamically linked, libltfs is compiled as a dynamic lib, the ICU messages bundles are compiled statically and linked into libltfs. libltfs should be a static lib?

[piste-jp]

Did you specify static link against resource bundle created by ICU?

It's dinamically linked, libltfs is compiled as a dynamic lib, the ICU messages bundles are compiled statically and linked into libltfs. libltfs should be a static lib?

No. The libltfs using in ltfs, mkltfs etc. is dynamic link library. What I'm saying is resource bundle created by ICU, like liblibltfs_dat.a is statically linked to a correct shared library.

For example, below is the build log of libltfs.so on auto tools. You can see ../../messages/liblibltfs_dat.a ../../messages/libinternal_error_dat.a ../../messages/libtape_common_dat.a is specified as object files. This command inject the message bundle into the libltfs.so

libtool: link: gcc -shared  -fPIC -DPIC  .libs/libltfs_la-ltfs.o .libs/libltfs_la-ltfs_internal.o .libs/libltfs_la-ltfs_fsops.o .libs/libltfs_la-ltfs_fsops_raw.o .libs/libltfs_la-ltfssnmp.o .libs/libltfs_la-fs.o .libs/libltfs_la-xml_common.o .libs/libltfs_la-xml_writer.o .libs/libltfs_la-xml_reader.o .libs/libltfs_la-xml_writer_libltfs.o .libs/libltfs_la-xml_reader_libltfs.o .libs/libltfs_la-label.o .libs/libltfs_la-base64.o .libs/libltfs_la-tape.o .libs/libltfs_la-iosched.o .libs/libltfs_la-dcache.o .libs/libltfs_la-kmi.o .libs/libltfs_la-pathname.o .libs/libltfs_la-index_criteria.o .libs/libltfs_la-xattr.o .libs/libltfs_la-ltfslogging.o .libs/libltfs_la-ltfstrace.o .libs/libltfs_la-ltfs_thread.o .libs/libltfs_la-config_file.o .libs/libltfs_la-plugin.o .libs/libltfs_la-periodic_sync.o .libs/libltfs_la-inc_journal.o arch/.libs/libltfs_la-uuid_internal.o arch/.libs/libltfs_la-filename_handling.o arch/.libs/libltfs_la-time_internal.o arch/.libs/libltfs_la-arch_info.o arch/.libs/libltfs_la-errormap.o   -ldl -lfuse -luuid -lxml2 -licui18n -licuuc -licudata -L/usr/lib/x86_64-linux-gnu -lnetsnmpmibs -lsensors -lpci -lnetsnmpagent -lwrap -lnetsnmp -lm -lssl -lcrypto ../../messages/liblibltfs_dat.a ../../messages/libinternal_error_dat.a ../../messages/libtape_common_dat.a -lrt -lpthread  -g -O2 -flto=auto -flto=auto -fstack-protector-strong -g -O2 -flto=auto -flto=auto -fstack-protector-strong -g -O2 -O0 -ggdb -Wl,--no-undefined -Wl,--as-needed -pthread -Wl,-Bsymbolic-functions -flto=auto -flto=auto -Wl,-z -Wl,relro -Wl,-z -Wl,now -Wl,-E -Wl,-Bsymbolic-functions -flto=auto -flto=auto -Wl,-z -Wl,relro -Wl,-z -Wl,now   -pthread -Wl,-soname -Wl,libltfs.so.0 -o .libs/libltfs.so.0.0.0

You can confirm the symbol table of libltfs.so. It must have resource bundle as value like below.

nm ./libltfs.so | grep _dat$
00000000000b20d0 R internal_error_dat
000000000009df90 R libltfs_dat
00000000000b9f00 R tape_common_dat

[madjesc]

No. The libltfs using in ltfs, mkltfs etc. is dynamic link library. What I'm saying is resource bundle created by ICU, like liblibltfs_dat.a is statically linked to a correct shared library.

For example, below is the build log of libltfs.so on auto tools. You can see ../../messages/liblibltfs_dat.a ../../messages/libinternal_error_dat.a ../../messages/libtape_common_dat.a is specified as object files. This command inject the message bundle into the libltfs.so

libtool: link: gcc -shared  -fPIC -DPIC  .libs/libltfs_la-ltfs.o .libs/libltfs_la-ltfs_internal.o .libs/libltfs_la-ltfs_fsops.o .libs/libltfs_la-ltfs_fsops_raw.o .libs/libltfs_la-ltfssnmp.o .libs/libltfs_la-fs.o .libs/libltfs_la-xml_common.o .libs/libltfs_la-xml_writer.o .libs/libltfs_la-xml_reader.o .libs/libltfs_la-xml_writer_libltfs.o .libs/libltfs_la-xml_reader_libltfs.o .libs/libltfs_la-label.o .libs/libltfs_la-base64.o .libs/libltfs_la-tape.o .libs/libltfs_la-iosched.o .libs/libltfs_la-dcache.o .libs/libltfs_la-kmi.o .libs/libltfs_la-pathname.o .libs/libltfs_la-index_criteria.o .libs/libltfs_la-xattr.o .libs/libltfs_la-ltfslogging.o .libs/libltfs_la-ltfstrace.o .libs/libltfs_la-ltfs_thread.o .libs/libltfs_la-config_file.o .libs/libltfs_la-plugin.o .libs/libltfs_la-periodic_sync.o .libs/libltfs_la-inc_journal.o arch/.libs/libltfs_la-uuid_internal.o arch/.libs/libltfs_la-filename_handling.o arch/.libs/libltfs_la-time_internal.o arch/.libs/libltfs_la-arch_info.o arch/.libs/libltfs_la-errormap.o   -ldl -lfuse -luuid -lxml2 -licui18n -licuuc -licudata -L/usr/lib/x86_64-linux-gnu -lnetsnmpmibs -lsensors -lpci -lnetsnmpagent -lwrap -lnetsnmp -lm -lssl -lcrypto ../../messages/liblibltfs_dat.a ../../messages/libinternal_error_dat.a ../../messages/libtape_common_dat.a -lrt -lpthread  -g -O2 -flto=auto -flto=auto -fstack-protector-strong -g -O2 -flto=auto -flto=auto -fstack-protector-strong -g -O2 -O0 -ggdb -Wl,--no-undefined -Wl,--as-needed -pthread -Wl,-Bsymbolic-functions -flto=auto -flto=auto -Wl,-z -Wl,relro -Wl,-z -Wl,now -Wl,-E -Wl,-Bsymbolic-functions -flto=auto -flto=auto -Wl,-z -Wl,relro -Wl,-z -Wl,now   -pthread -Wl,-soname -Wl,libltfs.so.0 -o .libs/libltfs.so.0.0.0

You can confirm the symbol table of libltfs.so. It must have resource bundle as value like below.

nm ./libltfs.so | grep _dat$
00000000000b20d0 R internal_error_dat
000000000009df90 R libltfs_dat
00000000000b9f00 R tape_common_dat

The symbols exists in the library

[mcardenas@kasane libltfs]$ nm ./libltfs.so | grep _dat$
00000000000959a0 R internal_error_dat
0000000000081830 R libltfs_dat
000000000009d800 R tape_common_dat

And cmake compiles it like this

/usr/bin/cc -fPIC -pthread -shared -Wl,-soname,libltfs.so -o libltfs.so CMakeFiles/libltfs.dir/arch/arch_info.c.o CMakeFiles/libltfs.dir/arch/errormap.c.o CMakeFiles/libltfs.dir/arch/filename_handling.c.o CMakeFiles/libltfs.dir/arch/osx/osx_string.c.o CMakeFiles/libltfs.dir/arch/time_internal.c.o CMakeFiles/libltfs.dir/arch/uuid_internal.c.o CMakeFiles/libltfs.dir/base64.c.o CMakeFiles/libltfs.dir/config_file.c.o CMakeFiles/libltfs.dir/dcache.c.o CMakeFiles/libltfs.dir/fs.c.o CMakeFiles/libltfs.dir/inc_journal.c.o CMakeFiles/libltfs.dir/index_criteria.c.o CMakeFiles/libltfs.dir/iosched.c.o CMakeFiles/libltfs.dir/kmi.c.o CMakeFiles/libltfs.dir/label.c.o CMakeFiles/libltfs.dir/ltfs.c.o CMakeFiles/libltfs.dir/ltfs_fsops.c.o CMakeFiles/libltfs.dir/ltfs_fsops_raw.c.o CMakeFiles/libltfs.dir/ltfs_internal.c.o CMakeFiles/libltfs.dir/ltfs_thread.c.o CMakeFiles/libltfs.dir/ltfslogging.c.o CMakeFiles/libltfs.dir/ltfssnmp.c.o CMakeFiles/libltfs.dir/ltfstrace.c.o CMakeFiles/libltfs.dir/pathname.c.o CMakeFiles/libltfs.dir/periodic_sync.c.o CMakeFiles/libltfs.dir/plugin.c.o CMakeFiles/libltfs.dir/tape.c.o CMakeFiles/libltfs.dir/xattr.c.o CMakeFiles/libltfs.dir/xml_common.c.o CMakeFiles/libltfs.dir/xml_reader.c.o CMakeFiles/libltfs.dir/xml_reader_libltfs.c.o CMakeFiles/libltfs.dir/xml_writer.c.o CMakeFiles/libltfs.dir/xml_writer_libltfs.c.o  /usr/lib64/libxml2.so ../../messages/libbin_ltfs.a ../../messages/liblibltfs.a ../../messages/libinternal_error.a ../../messages/libtape_common.a -lpthread /usr/lib64/libfuse.so /usr/lib64/libuuid.so 

[piste-jp]

I didn't try your code at all yet. But does auto tools build on your tree work fine? If so, you need to investigate step by step and find differences.

Please do not forget that I never ask you to do this. You just want to do this.

[madjesc]

I didn't try your code at all yet. But does auto tools build on your tree work fine? If so, you need to investigate step by step and find differences.

Please do not forget that I never ask you to do this. You just want to do this.

Yes, I know that I want to implement this, I'm not blaming anyone, I'm trying to get some advice for this. I'm trying to look at the differences between versions

[piste-jp]

I didn't try your code at all yet. But does auto tools build on your tree work fine? If so, you need to investigate step by step and find differences. Please do not forget that I never ask you to do this. You just want to do this.

Yes, I know that I want to implement this, I'm not blaming anyone, I'm trying to get some advice for this. I'm trying to look at the differences between versions

The message says error code retuned from ICU is 2. It means U_MISSING_RESOURCE_ERROR like below. ICU might not be able to find resource bundle into libltfs_dat.

LTFS11286E Cannot load messages: failed to open resource bundle (2)

    U_MISSING_RESOURCE_ERROR  =  2,     /**< The requested resource cannot be found */

The root cause is how to create message library file. In auto tools, .res files in the paths.txt does not include res_${NAME}. But it has res_$NAME/ and pkgdata generate resource name as libltfs/res_libltfs/en_US.res. In this resource bundle, the resource name shall be libltfs/res_libltfs, but code specifies it as libltfs. So ICU cannot find a resource bundle correctly. (PLEASE inspect the both .a files, you can see the resource names at the beginning of the files....)

Try the patch below.

diff --git a/.gitignore b/.gitignore
index 0711b11..e36512a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -23,6 +23,12 @@ ltfs.pc
 .libs/
 messages/.lib
 .dirstamp
+# Files renerated by CMake
+CMakeFiles
+*.cmake
+CMakeCache.txt
+install_manifest.txt
+*.so
 # Files created by OS
 .DS_Store
 # Files for development
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 5245b51..10130b2 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -1,8 +1,14 @@
 cmake_minimum_required(VERSION 3.20)
-project(ltfs,
+project(ltfs
   # TODO: Find a better solution to mark this version as different from the official build
   VERSION "2.5.0.9999"
+  LANGUAGES C
 )
+
+set(CMAKE_VERBOSE_MAKEFILE 1)
+set(CMAKE_C_COMPILER gcc)
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -ggdb -O0")
+
 include(CheckSymbolExists REQUIRED)
 include(CheckTypeSize REQUIRED)
 include(CheckLibraryExists REQUIRED)
diff --git a/messages/.gitignore b/messages/.gitignore
new file mode 100644
index 0000000..a445398
--- /dev/null
+++ b/messages/.gitignore
@@ -0,0 +1 @@
+res*
diff --git a/messages/CMakeLists.txt b/messages/CMakeLists.txt
index 46ef6c1..3b853b5 100644
--- a/messages/CMakeLists.txt
+++ b/messages/CMakeLists.txt
@@ -26,12 +26,13 @@ foreach(NAME IN LISTS MESSAGES_LIBS)
     OUTPUT "res_${NAME}/lib${NAME}.a"
     COMMAND mkdir -p res_${NAME}
     COMMAND ${ICU_GENRB_EXECUTABLE} -q ${SRCS} -d res_${NAME} 1> /dev/null
-    COMMAND find res_${NAME} -name *.res > res_${NAME}/paths.txt
-    COMMAND ${ICU_PKGDATA_EXECUTABLE} -d res_${NAME}/ -m static -p ${NAME} -q res_${NAME}/paths.txt 1> /dev/null
+    COMMAND cd res_${NAME} && ls *.res > paths.txt
+    COMMAND cd res_${NAME} && ${ICU_PKGDATA_EXECUTABLE} -m static -p ${NAME} -q paths.txt 1> /dev/null
     BYPRODUCTS "res_${NAME}"
   )
   # Add the command as a target to be build when all is invoked

Message resource problem is gone like below in my env.

[root@rocky9 ltfs-madjesc2]# ./src/ltfs
8c66 LTFS11268E Cannot open configuration file '/ltfs.conf' (-2).
8c66 LTFS10008E Failed to load the configuration file (-2).

[madjesc]

Sorry for the delay, I was occupied with other stuff. Whatever, now this works correctly, I tested it using the file backend and it works as expected

Please add a description how to build the tree with cmake even if it is experimental support.

Should I add it to the README?

[piste-jp]

Sorry for the delay, I was occupied with other stuff. Whatever, now this works correctly, I tested it using the file backend and it works as expected

Please add a description how to build the tree with cmake even if it is experimental support.

Should I add it to the README?

I think it is good to add a new document like BUILD_WITH_CMAKE.md and state this method is experimental at this time.