System libraries get installed during the install step

[sum01]

EDITED SUMMARY When linking directly with standard libraries they will get installed using the default install process. This should be avoided.

Also bsf data files should be placed in share/bsf instead of being in the bin folder.

EDIT no. 2: .dbg,.dwarf & .pdb files should not be installed by default for the release configuration. It is useful when packaging releases but a hassle for users that install directly in system folders.

ORIGINAL ISSUE Attempting to build Banshee Engine on Arch Linux, and I noticed that Bsf doesn't use any of the system dependencies. Posting here instead of the Banshee Engine issues since this is the source of the problem FindXXX files.

I know not all dependencies are on the repo's, but a majority are, and yet they aren't used. This makes compile times much longer than they need to be.

See this PKGBUILD for the build script

# Maintainer: sum01 <[email protected]>
pkgname=banshee-engine-git
pkgver=v0.4.0.r1116.gfd64e421f
pkgrel=1
pkgdesc='Modern C++14 game engine with Vulkan support, a fully featured editor, and C# scripting'
arch=('i686' 'x86_64')
url="https://github.com/BearishSun/BansheeEngine"
license=('LGPL3')
depends=('opengl-driver' 'libx11' 'libxcursor' 'libxrandr'
	'libutil-linux' 'bison>=3.0.4' 'flex' 'snappy'
	'freetype2>=2.3.5' 'freeimage>=3.13.1' 'openal>=1.17.2'
	'libogg>=1.3.2' 'libvorbis' 'flac' 'fmodex'
	'gtk3' 'mono>=5.4.0' 'python>=3.5.0' 'vulkan-validation-layers'
	'glslang' 'fbx-sdk' 'nvidia-texture-tools')
makedepends=('cmake>=3.9.0' 'make' 'git' 'sed')
source=("git+https://github.com/BearishSun/BansheeEngine.git")
sha512sums=('SKIP')
_dirname="BansheeEngine"
prepare() {
	cd "$srcdir/$_dirname"
	git submodule update --init --recursive
}
pkgver() {
	cd "$srcdir/$_dirname"
	git describe --long | sed 's/\([^-]*-g\)/r\1/;s/-/./g'
}
build() {
	mkdir -p "$srcdir/$_dirname/build"
	cd "$srcdir/$_dirname/build"
	# Install under /usr/lib instead of /usr/lib64
	cmake -DBUILD_TESTING=OFF -DCMAKE_BUILD_TYPE=Release \
		-DGENERATE_SCRIPT_BINDINGS=ON -DBUILD_BSL=ON \
		-DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_INSTALL_LIBDIR=lib ..
	make
}
package() {
	cd "$srcdir/$_dirname/build"
	make DESTDIR="$pkgdir" install
}

---

The only depdencies that weren't in the repo's (official & AUR) were the following:

• Physx
• XShaderCompiler
• BansheeSBGen

---

Sidenote: The Data folder is installed directly under /usr/bin/Data which is rather vague. I think installing to /usr/bin/banshee-engine/Data would be more preferrable (or any named folder).

[BearishSun]

Hi. When you say they don't work, what exactly is the end result?

FindXXX.cmake files should look for a library in the local Dependencies folder first, and if it cannot be found there, they should search all the default search paths such as /usr/local/lib. The default build process fetches the pre-compiled set of dependencies, so they will get found by default, unless you explicitly remove them (or disable this process in CMake in the first place).

Regarding the sidenote: At this point in time I didn't really expect Banshee to be installed in usr/bin, but rather to /opt in its own sub-folder. But if you feel like making the relevant changes to make it suitable to bin, I wouldn't mind.

Note that bsf/Banshee also uses slightly modified builds of some of the libraries: https://github.com/GameFoundry/bsf/blob/master/Documentation/GitHub/dependencies.md (note the non-official repos for custom versions). These are generally very minor modifications to the build, and could possibly be worked around differently so the libraries directly from the repo can be used instead.

[sum01]

Here's the resulting list of libs in the pkg.tar

I know some are for the project, but other like flac, vorbis, and openal should've been used from the system's libs.

[BearishSun]

FindXXX files register shared libraries for install whenever they find one. So it's quite likely those libraries you are seeing are copies of the system libraries (as long as your Dependencies folder is clear). In that case you should be able to remove them and everything should still work. Of course a proper fix would be to update FindXXX files to avoid registering a library for install in case its a system one.

[sum01]

Oh, didn't know they were cloned. Maybe that behaviour could be disabled with an option like option(BSF_INSTALL_FOUND_DEPS "Install found system dependencies." OFF)

As for the Data dir, you could include(GNUInstallDirs) and install to "${CMAKE_INSTALL_FULL_DATAROOTDIR}/bsf" or something of that nature.

I guess I could dig through some of the files and try to make those changes if you wanted.

[BearishSun]

I'll get around to it eventually, but if you want it done soon it's best you take a look yourself.

You can probably skip the install step on a per-library basis, rather than disabling it globally. All the FindXXX files are fairly small and make use of the helper macros in Source/CMake/HelperMethods.cmake. That's a good point to start at (find_library calls with NO_DEFAULT_PATH parameter look locally, while ones without it look in system folders).

Installing data to share/bsf sounds good. I wouldn't use the absolute path so users can still install to custom locations. It will need to be changed in the code itself, look for usages of FRAMEWORK_DATA_PATH.

[sum01]

CMAKE_INSTALL_FULL_DATAROOTDIR is settable by the end-user, it looks like cmake -DCMAKE_INSTALL_DATAROOTDIR=someotherdir ...

FULL_ is shorthand instead of having to do ${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_DATAROOTDIR}. All the GNUInstallDirs variables work like that.

---

I guess I'll rm some of those libs in the package() step, and push it to the AUR later.

[sum01]

So I looked into it, and "built-in" methods are available to get a majority of the dependencies without custom Findxxx modules.

Using a mix of PkgConfig, which Cmake integrates nicely, upstream-provided xxxConfig/Findxxx files, and official Findxxx modules packaged with Cmake, you can get the following libs:

Lib	Provided By
FLAC	PkgConfig
ICU	FindICU
uuid	PkgConfig
OpenAL	FindOpenAL
Vulkan	FindVulkan
XShaderCompiler	Upstream FindXShaderCompiler (${XSC_LIBRARY} & ${XSC_INCLUDE_DIR})
BISON	FindBISON
FLEX	FindFLEX
Freetype	FindFreetype
Mono	PkgConfig
Ogg	PkgConfig
Snappy	Upstream SnappyConfig (imports lib Snappy::snappy)
Vorbis	PkgConfig

The only issue is this doesn't automatically download missing libs like you currently have it do. I guess if you wanted to keep that functionality, ExternalProject could be used.

[BearishSun]

The main problem with the built-in find package calls is that they do not look for the dependency locally, but instead always look in system folders. bsf expects the search to happen first locally, and only look in system folders if that fails, to ensure users can link using exact version of their dependencies.

[sum01]

I mostly fixed this issue, but for some reason OpenAL still gets installed and I can't seem to find the reason why. Also, is there any reason for installing .dbg files on a release build?

Sidenote: The Cmake files are a big pile of spaghetti. Also, you're mixing old and "modern" Cmake while requiring Cmake >= v3.9. It's weird :confused:

[BearishSun]

Sounds good. The .dbg files are purposefully generated and installed in release builds so they can be saved before the release is packaged and distributed. This way error reports from people running release code with no debug symbols can be debugged. Otherwise you are running blind when the user sends you a callstack containing only addresses.

If you have a suggestion on how to organize the files let me know, otherwise I feel they are better than most CMake files I've seen out there - but that's not saying much as CMake is far from a clean language. Also I'm not aware what is considered old CMake, but let me know and I'll see if it can be modernized.

[sum01]

The main issue with the .dbg files is that they get installed to /usr/lib during an install from source. Is there perhaps a more suitable place to install them? Or would it makes sense to create a /usr/lib/bsf and dump everything in there?

When I mentioned spaghetti, I meant more as to how there's macros running macros running macros (find_imported_library for example).

The old/modern thing is nitpicky, but it's things like messing with variables like CMAKE_XXX_FLAGS, or using global settings for things that are usually better off as a target-specific call like target_xxx_xxx(), or using that Findbsf module instead of exports. Properties.cmake is an example of old Cmake idioms.

[BearishSun]

Indeed, that's an issue. We could perhaps add a property to toggle install of debug symbol files and make it off by default. Currently I only require this functionality for CI and when releasing official versions of the framework.

I'd prefer not to put bsf in its own folder (although that would solve the issue with Data files as well - but I want to eventually move that to /share/bsf anyway).

Are there any issues that could arise from macros running macros? What would be the preferable alternative?

I agree about the lack of use target_* calls. I'll be updating that when I get a chance, and ditching the global flags.

What would be the alternative to Findbsf? I was under the impression that exports only handle the case when other CMake projects link with source code of another CMake project, while find_package is used for locating binaries. But my knowledge is lacking in this area.

[sum01]

Exports can, if installed correctly (in the right place), be used for find_package calls. They correctly define public/private/interface depedencies that are required to link to it. They are similar to PkgConfig's .pc files, but for Cmake.

Along with using a Config.cmake.in file with the CMakePackageConfigHelpers macros, you can have a find_package(bsf 1.2.3 REQUIRED) esque call, along with Semver versioning (or however you choose to do versioning). All of that is generated on each build so you aren't making constant changes to keep it up-to-date.

If you want an example of this, look at Catch2, which when doing find_package(Catch2) will import the lib Catch2::Catch. I've done the same thing in this smaller project which might be easier to read since it's roughly a 100 LOC CMakeLists.txt

---

The thing with the macros running macros etc. was it's just hard to find what happens where, no "actual" problems arise from that. :smile:

Sidenote: If you use option(BUILD_SHARED_LIBS "Build shared instead of static" OFF) then remove the STATIC or SHARED from all add_library calls, it lets the user define a static or shared build.

[BearishSun]

Exports look interesting, thanks, I'll probably add them at some point when I have more free time.

[sum01]

Forgot I worked on partially fixing this, so the patch is a bit old, but here's most of the issue solved.

partial_fix.diff.gz

Sadly it still has the problem of OpenAL being installed, but everything else is fixed. I wasn't able to see what I was missing that makes OpenAL get installed, so maybe someone else will.