Cannot compile with CMake on macOS 12.2.1 if option `CURL_USE_GSSAPI` is On

[kuklinistvan]

I did this

I attempted to compile Curl in a CMake project by adding the repository as a subproject. After seemingly successful configuration I've encountered the following error:

/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_in_addr_t.h:31:25: error: 'type-name' cannot be signed or unsigned
typedef __uint32_t      in_addr_t;      /* base type for internet address */
                        ^
/Users/ikuklin/Desktop/BMOD_4310_src/curl_api_client_build/deps/curl/lib/curl_config.h:994:19: note: expanded from macro 'in_addr_t'
#define in_addr_t unsigned long

I began investigating and I traced back the issue as follows:

1. I noticed a high amount of platform checking tests failing on which I first thought Xcode was not correctly installed. Among them there were these lines:

-- Performing Curl Test HAVE_IN_ADDR_T
-- Performing Curl Test HAVE_IN_ADDR_T - Failed

If my perception is right then these are done to ensure compatibility among the supported platforms and provide the missing features through the generated portions of curl_config.h in case if something is missing. In my case in_addr_t was falsely assumed missing and therefore a define was generated to aid that which clashed with the already existing in_addr_t type on my system.

2. I examined how CurlTests.c and curl_internal_test macro works to find out why that happened. For context initially I set CURL_USE_GSSAPI On for the reason the version of curl forked by Apple suggested so in README.Apple inside the tarball.

I printed out in Macros.cmake at try_compile() call MACRO_CHECK_FUNCTION_DEFINITIONS

    try_compile(${CURL_TEST}
      ${CMAKE_BINARY_DIR}
      ${CMAKE_CURRENT_SOURCE_DIR}/CMake/CurlTests.c
      CMAKE_FLAGS -DCOMPILE_DEFINITIONS:STRING=${MACRO_CHECK_FUNCTION_DEFINITIONS}
      "${CURL_TEST_ADD_LIBRARIES}"
      OUTPUT_VARIABLE OUTPUT)

and it looked like so (notice the end):

-- -DCOMPILE_DEFINITIONS:STRING=-DHAVE_IN_ADDR_T  -DHAVE_LDAP_H -DHAVE_LBER_H -DHAVE_GSSAPI_GSSAPI_H -DHAVE_GSSAPI_GSSAPI_GENERIC_H -DHAVE_GSSAPI_GSSAPI_KRB5_H -DHAVE_INTTYPES_H -DHAVE_SYS_SOCKET_H -DHAVE_SYS_TIME_H -DHAVE_SYS_TYPES_H -DHAVE_ARPA_INET_H -DHAVE_FCNTL_H -DHAVE_NETDB_H -DHAVE_NETINET_IN_H -DHAVE_PWD_H -DHAVE_STDINT_H -DHAVE_TIME_H -DHAVE_UNISTD_H -DHAVE_STDDEF_H -DHAVE_STDINT_H  -dynamic -Wl,-search_paths_first

Since this is passed to argument CMAKE_FLAGS of try_compile() it turns out that it fails (silently I think) and instead of skipping the one garbage part that it cannot handle (-dynamic -Wl,-search_paths_first) or cause a fatal error, it just omits the compiler flags.

To prove this I intentionally caused a fatal error just after the test HAVE_IN_ADDR_T failed and with --debug-trycompile enabled and examined the build environment for the feature test. None of the defines were passed over to the compiler. After I quickly have found a way to prevent the part -dynamic -Wl,-search_paths_first being passed to CMAKE_FLAGS not only the test passed but the whole build process has healed itself.

3. With variable_watch I traced back where -dynamic -Wl,-search_paths_first came from:

• Into CMAKE_FLAGS it was passed through the variable ${MACRO_CHECK_FUNCTION_DEFINITIONS},
• a couple lines earlier it was concatenated from ${CMAKE_REQUIRED_FLAGS},

      foreach(_dir ${GSS_LINK_DIRECTORIES})
        set(_LINKER_FLAGS_STR "${_LINKER_FLAGS_STR} -L\"${_dir}\"")
      endforeach()

      set(CMAKE_REQUIRED_FLAGS "${_COMPILER_FLAGS_STR} ${_LINKER_FLAGS_STR}")

• which was set in the top CMakeLists.txt by concatenating ${_LINKER_FLAGS_STR},

      string(REPLACE ";" " " _COMPILER_FLAGS_STR "${GSS_COMPILER_FLAGS}")
      string(REPLACE ";" " " _LINKER_FLAGS_STR "${GSS_LINKER_FLAGS}")

• which came from {GSS_LINKER_FLAGS},
• that is set by FindGSS.

curl/libcurl version

It both affects master at the time of writing (134963a5efdc3906257c88ce62dba8d46c292908) and tag curl-7_83_1.

operating system

Darwin MYHOSTNAMEREMOVED 21.3.0 Darwin Kernel Version 21.3.0: Wed Jan 5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_X86_64 x86_64

macOS 12.2.1 Monterey

Unfortunately in a couple days I'll no longer have access to this computer so I did my best assemble all the information that should be enough to reproduce the issue.

[jzakrzewski]

I suspect this is the same problem I patch locally when building curl - I haven't upstreamed that because I had no time to check if it has any unwanted side effects.

I just add string(REPLACE ";" " " GSS_LINKER_FLAGS "${GSS_LINKER_FLAGS}") after line 862. I am almost sure this is correct. Almost...

[kuklinistvan]

@jzakrzewski I've stumbled upon this issue again on another computer and have tried your workaround. It seems to be working.

[bagder]

@jzakrzewski do you want to create a PR out of your suggestion so we can have this fixed?

[jzakrzewski]

I'll try to make some time either this evening or tomorrow. I'd really love to contribute more than occasional comment here and there...

[jzakrzewski]

Damn, still did not get to it. :( I'm doing something wrong with my time management...