Adapt vcpkg patches and update vcpkg port to 4.5

[kelteseth]

Godot version

4.5

godot-cpp version

4.5

System information

Windows 11

Issue description

Currently, the vcpkg version of godot-cpp is at 4.4. With a custom install step that is needed by vcpkg. We should integrate these changes into godot-cpp directly. I'm currently working on porting our custom godot-cpp-vcpkg fork to 4.5 and I'm facing some challanges since there have been some changes regarding CMake generator expression. I would like to update the vcpkg port, but I need some feedback first:

The current 4.4 patch alters several files, but I have moved the install code into a dedicated file and included it at the end of the main CMakeLists.txt via include("cmake/install.cmake") to make it easier to read.

My install.cmake now uses generator expressions

include(GNUInstallDirs)
include(CMakePackageConfigHelpers)

# Install the godot-cpp target
install(TARGETS godot-cpp
    EXPORT unofficial-godot-cpp-config
    ARCHIVE DESTINATION "${CMAKE_INSTALL_LIBDIR}"
    LIBRARY DESTINATION "${CMAKE_INSTALL_LIBDIR}"
    RUNTIME DESTINATION "${CMAKE_INSTALL_BINDIR}"
)

install(
    DIRECTORY
        "${CMAKE_CURRENT_SOURCE_DIR}/include/"
        "${CMAKE_CURRENT_BINARY_DIR}/gen/include/"
    DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}"
)

install(FILES "${GODOTCPP_GDEXTENSION_DIR}/gdextension_interface.h"
    DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}"
)

install(
    EXPORT unofficial-godot-cpp-config
    NAMESPACE unofficial::
    DESTINATION "${CMAKE_INSTALL_DATADIR}/unofficial-godot-cpp"
)

write_basic_package_version_file(
    "${CMAKE_CURRENT_BINARY_DIR}/unofficial-godot-cpp-config-version.cmake"
    VERSION "${PROJECT_VERSION}"
    COMPATIBILITY SameMinorVersion
)

install(
    FILES "${CMAKE_CURRENT_BINARY_DIR}/unofficial-godot-cpp-config-version.cmake"
    DESTINATION "${CMAKE_INSTALL_DATADIR}/unofficial-godot-cpp"
)

The current 4.4 Portfile does not explicitly set GODOTCPP_TARGET, so I added:

vcpkg_cmake_configure(
    SOURCE_PATH "${SOURCE_PATH}"
    OPTIONS
        "-DPython3_EXECUTABLE=${PYTHON3}"
    OPTIONS_DEBUG
        -DGODOTCPP_TARGET=template_debug
    OPTIONS_RELEASE
        -DGODOTCPP_TARGET=template_release
)

Not setting this resulted in 2 issues:

1. File name was debug instead of release:vcpkg-src\buildtrees\godot-cpp\x64-windows-rel\bin\libgodot-cpp.windows.template_debug.x86_64.lib
2. File size was different, that is an indicator that DEBUG_METHODS_ENABLED where enabled even if they should not be.

I faced similar issues like github.com/godotengine/godot/issues/64897 where my extension worked fine in debug but crashed on exit in release (Note we use Godot as embedded Library, so there are also some differences there).

Steps to reproduce

Minimal reproduction project

[Ivorforce]

Hi, we've discussed vcpkg before. The godot-cpp team is currently not convinced that we want to support user-wide godot-cpp installations, because we think godot-cpp installations should be separate per project.

See also https://github.com/godotengine/godot-cpp/pull/1418 (which, now that I see it again, we never commented with this assessment?)

[kelteseth]

You do not have to have vcpkg packages installed globally with vcpkg. We have a build vcpkg script that compiles vcpkg once and then zips the vcpkg/installed folder for all other developers to use, then you can simply include the prebuild vcpkg via "CMAKE_TOOLCHAIN_FILE": "${sourceDir}/../vcpkg/scripts/buildsystems/vcpkg.cmake" . We had godot-cpp initially included directly as submodule, but this killed our build times, so we moved it to vcpkg.

Installed here only means inside vcpkg, not system-wide.

[enetheru]

@kelteseth I havent looked into installation in detail due to the discouragement of system installation and never having had the need. But I've seen it utilised in practice a bunch of times like you describe.

Is there a way of implementing this that discourages package managers from trying to include it into package trees, or users wanting to install it system wide?

Also you should look into using the build profile feature if so you aren't generating and including the kitchen sink( if you arent actually using it all)

[Ivorforce]

If build times are your main concern, that should not be a problem if you use incremental builds. For local development, godot-cpp really only needs to build once, and then your local files are the only ones that need recompilation on changes. If you're using CI, it might be useful to look into a caching solution. If vcpkg reduces your build times, that's exactly what it is doing, too. godot-cpp's own CI, as well as godot-cpp-template, both have caching set up with GitHub runners.

[kelteseth]

With CMake (and especially Qt) we often have to rebuild everything. Even just adding a simple .qml file or icon, means recompiling everything. This is nothing we can do here, so no this is not a valid solution for us :(

discourages package managers from trying to include it into package trees,

Is this about Linux package maintainers? A simple warning in the Readme about that godot-cpp should never be used like apt install godot-cpp should be enough?

[Ivorforce]

Adding a new .qml file shouldn't affect godot-cpp. godot-cpp is its own static library that shouldn't even know your Qt files exist. I'd be surprised if CMake was any different there from SCons (@enetheru correct me if I'm wrong). What triggers the rebuild?

[enetheru]

What triggers the rebuild?

If the cmake is causing rebuilding unnecessarily I would like to know about it.

[kelteseth]

We are getting off-topic here about Qt CMake internals (Like often it is that CMake has some nasty caching issues where you have to nuke your build folder, just for the change to take effect) and I guess that was a bad example by me.

So again I have a c++ project with many dependencies, I can manage every single dependency I have with vcpkg to reduce compile times, why should godot-cpp here be different? 🙂

[Ivorforce]

We are getting off-topic here about Qt CMake internals (Like often it is that CMake has some nasty caching issues where you have to nuke your build folder, just for the change to take effect) and I guess that was a bad example by me.

It should be possible to modify your CMakeLists.txt to isolate godot-cpp from the rest, so it won't have invalid caches as the rest invalidate. Like I said, it's a separate static library, so it should not be affected by your project at all. If you're facing issues, maybe @enetheru can help?

So again I have a c++ project with many dependencies, I can manage every single dependency I have with vcpkg to reduce compile times, why should godot-cpp here be different? 🙂

The vcpkg entry isn't hosted or maintained by us (and like I said, we aren't planning to support it). Maybe you'd like to propose these changes to the vcpkg entry you linked instead?

[enetheru]

It should be possible to modify your CMakeLists.txt to isolate godot-cpp from the rest, so it cannot have invalid caches as the rest invalidate.

@Ivorforce The way cmake handles transient dependency information( even though godot-cpp doesnt have any ), is using the install files and configure scripts that are generated on install hey. Which is why people want installable libraries, they get consumed as external libraries from some location. there is a nice video i have in my library about it here: https://youtu.be/nBptg3SHPGU?si=aXjPhgJZNkLjPI0V

Like I said, it's a separate static library, so it should not be affected by your project at all. If you're facing issues, maybe @enetheru can help?

Indeed.

(Like often it is that CMake has some nasty caching issues where you have to nuke your build folder, just for the change to take effect)

I've honestly never had any issues, nor read about issues in regards to the cmakecache, so if you have any problems with godot-cpp rebuilding unnecessarily please let me know.

[kelteseth]

Is there a way of implementing this that discourages package managers from trying to include it into package trees, or users wanting to install it system wide?

Maybe something like this

if(CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT OR 
   CMAKE_INSTALL_PREFIX MATCHES "^(/usr|/usr/local|/opt)")
    message(WARNING 
        "godot-cpp is NOT intended for system-wide installation.\n"
        "Each project should use its own godot-cpp version.\n"
        "Consider using a local prefix or a package manager like vcpkg.")
endif()

or

option(GODOTCPP_ENABLE_INSTALL "Enable install targets (discouraged for system-wide use)" OFF)

if(GODOTCPP_ENABLE_INSTALL)
    include(cmake/install.cmake)
endif()

and a simple note in the projects Readme:

⚠️ Do NOT install godot-cpp system-wide (via apt/pacman/brew). Each project must use its own version.

The Godot vcpkg port is already labeled unofficial, and I think this is also a good sign for anyone that this is not meant for traditional system-wide package management. The unofficial:: namespace clearly states: "use at your own risk / not officially supported.

[enetheru]

I have more questions while we're at it. Using straight CMake both FetchContent or ExternalProject end upp behaving like a source package that provides the opportunity to configure before building and consuming. Does vcpkg also support source packages, and if so could it be designed to present as a source package publicly, with the option for local in house maintainers like yourself to provide binary packages for your colleagues?

I think having rudimentary support for a source package like thing would actually reduce the number of binary distributors and do more to discourage its use than nothing at all.

[kelteseth]

Yeah, vcpkg is actually primarily a source package manager - it builds from source by default. The binary caching is just an optional performance optimization.

[enetheru]

Well i'll try remember to raise it in the next meeting if i attend, see if there can be a pathway forward that satisfies all parties.

[dsnopek]

All the suggestions for discouraging system-wide installation in the comments above sound pretty good to me

Installed here only means inside vcpkg, not system-wide.

We also want to discourage "user-wide" installations (although, not as strongly as "system-wide" since there are valid use cases here), and encourage folks to use a build_profile.json for each specific project - that leads to faster build times (compared to a full build, but not as fast as a prebuilt static library) and much smaller binaries. Without using a build_profile.json, even an "empty" extension ends being like ~20mb

Yeah, vcpkg is actually primarily a source package manager - it builds from source by default.

Is this building from source for the specific project, and does it allow configuring its build options for the specific project?

If so, and folks can use vcpkg in combination with a unique build_profile.json for their project, that actually sounds great :-)

[myblindy]

There are some misconceptions about vcpkg here: it is very simple to have project-specific vcpkg libraries that are compiled inside and linked from your project folder only by using a vcpkg.json manifest file (example). That also includes specifying specific version numbers, or features, that don't have to match other links on the system.

It is also easy to include project-specific source file patches as needed using overlay ports (example).

Don't sleep on this package manager, it has everything you might need included out of the box!

[dsnopek]

Could someone share the exact steps that you would follow to use vcpkg to include godot-cpp in your project and compile it with project specific options (most importantly a custom build_profile.json)?

We don't have much experience with vcpkg and seeing what the process looks like would be super helpful for us to understand how it works in the context of what's important for godot-cpp