CPM.cmake Problems including SDL

I'm trying to use CPM in conjunction with https://github.com/libsdl-org/SDL and it is quite a wild ride. As per official practice you include the modules with #include <SDL2/SDL_xxxxxx.h> which matches how SDL presents the inclusions when built & installed.

Unfortunately however, SDL's inclusions folders in the source code do not match this exported pattern. The headers are inside the naked include folder in the root of the source tree.

This breaks code inclusion with CPM because the headers are expected in a different folder. I can include it with #include <SDL_xxxxxx.h> but then it will break when it finds SDL already installed on the user's system.

I'm searching through ALL the samples included in CPM, and all the libraries use the proper/consistent folder pattern. Is there a way already built into CPM to combat this discrepancy that I can leverage? Or am I stuck having to do some hacky magic with folders myself?

Jul 15 '22 19:07 Banderi

Do you intend to build the library itself as well ? (By the fact you want to use the provided headers I must assume yes).

To me this is more of a CMake issue than a CPM specific issue, because you'd have that issue regardless of how you included SDL2's source.

The best way for you IMO to achieve what you want is to add SDL2 with CPM as you're currently doing, but then also add an operation to copy the headers into your build directory (e.g. <binary_dir>/somesubdir/SDL2/...), likely as a custom target that you'd create yourself and you'd make as a dependency of the SDL2 library target. And this is not convoluted at all, SDL2 does it for itself ! Just look at the sdl_headers_copy target in their CMakeLists.txt and see for yourself !

Then what you'd want to do is make that directory an include directory of the SDL2 target via target_include_directories (as an INTERFACE property so as not to tamper with the build of SDL2 itself).

Good luck !

Jul 17 '22 21:07 oddko

Hey there! Thanks much for the inputs!

Out of curiosity, how do libraries normally work with CPM? I thought CPM already fetched the library's source code and built it regardless of circumstances, or is that incorrect? (unless system installed libs are enabled and it finds one, of course)

I'll mention I'm a complete noob with CMake, I've been having the worse headache of my life trying to make things work with it... Which is really why I was looking into things like CPM hoping it would be a "add and forget" sort of deal (and the most part, it seems so! unfortunately SDL2 just so happens to not follow the same convention as the other libs)

Since literally all the sample libs work as expected, I think this is a problem of SDL2 doing things in a convoluted/non-conformant way rather than CPM's fault, but then again if I have understood correctly, there really isn't any way to do things "correctly" with CMake? And so if I brought it up with the SDL team, surely they'd tell me to go pound sand.

Would there be, maybe, a way in C++ to include libraries in a conditional manner? i.e. include the same file SDL.h but "try" to include it both with <SDL2/SDL.h> and <SDL.h>, and just pick the correct one automatically? Maybe with C++ macros or something similar?

If there isn't, I think my best option would be to just disregard the "proper" convention with SDL2 inclusion and just manually add a inclusion for the .../SDL2 subfolder of system-installed SDL2, if detected by CMake/CPM, so that #include <SDL.h> will always work no matter what... I really really really want to avoid messing with the source folders/copying headers/adding build targets etc. as to avoid more CMake headaches, and add any non-library-agnostic complexity, as much as I can. But, of course, there might also be good reasons for why this is a terrible idea and doing as you say is the only way to do things properly. If that's the case, I guess I'll swallow the bitterness and try to make that work, haha...

Jul 18 '22 09:07 Banderi

What CPM does is fetching the repository and loading its CMakeLists.txt if present. You'd get the exact same behavior cloning it somewhere on your computer and using add_subdirectory on it. CPM is just a neat dependency management tool that helps you easily add or remove modules and control the versions being used. Otherwise it just clones and adds pretty much.

As far as libraries like SDL2 go, they're intended to be built and installed on your system (e.g. in /usr/lib and /usr/include) rather than brought into a self-contained build. It's not that it's a wrong use of it it's perfectly fine, but the code as is aims for an install of the library and headers. So you indeed hit that wall when wanting to use it directly within your project.

C++ way of selecting headers ? That seems a bit of a hassle for "my headers are not in the right directory", however checking for the existence of include files would probably involve once again... you guessed it... CMake, which can do such things. It's then just a matter of setting a compiler macro on your target or whatever.

My best advice is as I said to tweak/add to the library project file because most of these big projects aren't really intended to be used that way. It's usually not a huge fix to do and I'm sure you can achieve it because in your case it's just a matter of copying these files in a suitable directory structure (i.e. some SDL2 subdirectory) and including the directory just above that.

Jul 18 '22 21:07 oddko

I see, that makes much sense. Thanks!

For the C++ route I found GCC has the __has_include directive which is incredibly useful, and it does exactly what I need but sadly it's not supported by all compilers. I tried to get CMake to find inclusion files, but to no avail either.

Will try to do as you recommended, hopefully it won't be too much of a nightmare... Thanks for the help!

Jul 19 '22 05:07 Banderi

So, I've read through the SDL cmakelists.txt and this seems to be the code that enumerates the headers:

# This list holds all generated headers.
# To avoid generating them twice, these are added to a dummy target on which all sdl targets depend.
set(SDL_GENERATED_HEADERS)

# ...

# Copy all non-generated headers to "${SDL2_BINARY_DIR}/include"
# This is done to avoid the inclusion of a pre-generated SDL_config.h
file(GLOB SDL2_INCLUDE_FILES ${SDL2_SOURCE_DIR}/include/*.h)
set(SDL2_COPIED_INCLUDE_FILES)
foreach(_hdr IN LISTS SDL2_INCLUDE_FILES)
    if(_hdr MATCHES ".*(SDL_config|SDL_revision).*")
        list(REMOVE_ITEM SDL2_INCLUDE_FILES "${_hdr}")
    else()
        get_filename_component(_name "${_hdr}" NAME)
        set(_bin_hdr "${SDL2_BINARY_DIR}/include/${_name}")
        list(APPEND SDL2_COPIED_INCLUDE_FILES "${_bin_hdr}")
        add_custom_command(OUTPUT "${_bin_hdr}"
            COMMAND ${CMAKE_COMMAND} -E copy_if_different "${_hdr}" "${_bin_hdr}"
            DEPENDS "${_hdr}")
    endif()
endforeach()
list(APPEND SDL_GENERATED_HEADERS ${SDL2_COPIED_INCLUDE_FILES})
PrintList(SDL_GENERATED_HEADERS "SDL_GENERATED_HEADERS:")

# ...

# Create target that collects all all generated include files.
add_custom_target(sdl_headers_copy
    DEPENDS ${SDL_GENERATED_HEADERS})

And I tried to distill it into this:

if(DEFINED SDL2_SOURCE_DIR)
    message("# Since SDL2 was imported and build from source code, we try to \"soft install\" the header files ourselves...")
    message("# Headers folder set to: ${SDL2_BINARY_DIR}/include/SDL2")
    SET(SDL_HEADERS_SOFT_INSTALL_DIR ${SDL2_BINARY_DIR}/include/SDL2)
    # Copy all non-generated headers to "${SDL2_BINARY_DIR}/include/SDL2"
    # This is done to avoid the inclusion of a pre-generated SDL_config.h
    file(GLOB SDL2_INCLUDE_FILES ${SDL2_SOURCE_DIR}/include/*.h)
    set(SDL2_COPIED_INCLUDE_FILES)
    foreach(_hdr IN LISTS SDL2_INCLUDE_FILES)
        if(_hdr MATCHES ".*(SDL_config|SDL_revision).*")
            list(REMOVE_ITEM SDL2_INCLUDE_FILES "${_hdr}")
        else()
            get_filename_component(_name "${_hdr}" NAME)
            set(_bin_hdr "${SDL_HEADERS_SOFT_INSTALL_DIR}/${_name}")
            list(APPEND SDL2_COPIED_INCLUDE_FILES "${_bin_hdr}")
            add_custom_command(OUTPUT "${_bin_hdr}"
                COMMAND ${CMAKE_COMMAND} -E cmake_echo_color --cyan "Copying ${_name} to ${SDL_HEADERS_SOFT_INSTALL_DIR}..."
                COMMAND ${CMAKE_COMMAND} -E copy_if_different "${_hdr}" "${_bin_hdr}"
                DEPENDS "${_hdr}")
        endif()
    endforeach()

    # Create target that collects all all generated include files.
    add_custom_target(sdl_headers_copy
        DEPENDS ${SDL2_COPIED_INCLUDE_FILES})
    # Add dependency and inclusion dirs to SDL2
    add_dependencies(SDL2 sdl_headers_copy)
    target_include_directories(SDL2 INTERFACE ${SDL2_BINARY_DIR}/include)
endif()

And so if understand correctly:

We declare a custom target (sdl_headers_copy) which depends on the SDL_GENERATED_HEADERS files
We declare said files as a custom output via the add_custom_command(OUTPUT ... ) directives
We add the dependency of the custom target to the primary SDL2 library
We add the folder as an include path to the primary SDL2 library (as INTERFACE)

However, this doesn't seem to work... I'm getting this error:

Target "SDL2" INTERFACE_INCLUDE_DIRECTORIES property contains path:

    "E:/Git/Antimony-III/cmake-build-release/_deps/sdl-build/include"

  which is prefixed in the build directory.

Instead of adding the dependency to SDL2, I tried adding it to my main project to test if it works, and although it runs, it doesn't output any files. It does output the files upon building the project, but I want the files to be available after the configuration step, if possible; I tried doing this:

    foreach(_hdr IN LISTS SDL2_INCLUDE_FILES)
        if(_hdr MATCHES ".*(SDL_config|SDL_revision).*")
            list(REMOVE_ITEM SDL2_INCLUDE_FILES "${_hdr}")
        else()
            get_filename_component(_name "${_hdr}" NAME)
            message("Copying ${_name} to ${SDL_HEADERS_SOFT_INSTALL_DIR}...")
            FILE(COPY ${_hdr} DESTINATION ${SDL_HEADERS_SOFT_INSTALL_DIR})
        endif()
    endforeach()

Which seems to work, but if I understand correctly this doesn't have the advantage of copy_if_different, is that correct?

Jul 21 '22 02:07 Banderi

Yes this will likely run everytime you configure CMake, I gave it a quick go and came up with this:

CPMAddPackage(GITHUB_REPOSITORY libsdl-org/SDL
        GIT_TAG main
        OPTIONS
            "SDL2_DISABLE_INSTALL TRUE")

if (SDL_ADDED)
    file(GLOB SDL_HEADERS "${SDL_SOURCE_DIR}/include/*.h")

    # Create a target that copies headers at build time, when they change
    add_custom_target(sdl_copy_headers_in_build_dir
            COMMAND ${CMAKE_COMMAND} -E copy_directory "${SDL_SOURCE_DIR}/include" "${CMAKE_BINARY_DIR}/SDLHeaders/SDL2"
            DEPENDS ${SDL_HEADERS})

    # Make SDL depend from it
    add_dependencies(SDL2 sdl_copy_headers_in_build_dir)

    # And add the directory where headers have been copied as an interface include dir
    target_include_directories(SDL2 INTERFACE "${CMAKE_BINARY_DIR}/SDLHeaders")
endif()

Note the disabling of installs of SDL2 targets, as CMake does not allow INTERFACE headers that are present in the build directory on targets that are to be installed with their headers (which is the error you got), since these are to be installed as well.

Jul 21 '22 12:07 oddko

I've tried the above, but unfortunately it doesn't work... it still only copies the files over when building, not when configuring. Also, doesn't copy_directory lack the benefit of copy_if_different, too?

Regarding that error, I would have never ever found out the SDL2_DISABLE_INSTALL thing on my own, haha. There's no universe where I would have stumbled across that. All of these things are just random esoteric noise and everything is hours and hours of trial-and-error with CMake.. 😅 Is there any particular reason we we need to make SDL2 dependent on this target and not the other way around?

(I apologize by the way if this is clearly outside the topic of CPM, thanks again for the help so far!)

Jul 23 '22 01:07 Banderi

I've tried the above, but unfortunately it doesn't work... it still only copies the files over when building, not when configuring.

This is the intended behavior though. While this is a basic header copy, I would still refer to it as a codegen step, and you do not want to accumulate these in the configure stages or else modifying project files will become a pain because they will take a while to run (and you're on Windows the worst in terms of filesystem performance). While what you're doing is probably a personal project and these are overblown concerns, I'd still start getting the habit of making these build steps.

Not only does this avoid long configure times, it also enables you to only do the codegen steps you really need (in that case, when you build something that links to SDL2), and make these codegen steps depend on input files, in this case the header files of the SDL repo (see the use of DEPENDS), hence why the copy_if_differentoptimization is kind of useless in this case since it will never re-run unless you change SDL sources. It is made a dependency of SDL2 rather than the flipside because you link against the library, not the custom target, and if the custom target goes unused it will never run, thus not copy your headers.

The pattern of having a <project_name>_INSTALL or whatever variable is a common one in open-source projects to enable the users of these librairies to disable the provided install steps which are as discussed mostly targetted for a system-wide install of the compiled librairies.

Jul 23 '22 10:07 oddko