Since CMake 3.24 we can inject logic into find_package commands. The idea here is to run our own package manager which will then compile dependencies on demand.

Code-Reviewer Section

The general pull request guidelines can be found here.

Please check each of the following things and check all boxes before accepting a PR.

[ ] The PR has a description, explaining both the problem and the solution.
[ ] The description mentions which forms of testing were done and the testing seems reasonable.
[ ] Every function/class/actor that was touched is reasonably well documented.

For Release-Branches

If this PR is made against a release-branch, please also check the following:

[ ] This change/bugfix is a cherry-pick from the next younger branch (younger release-branch or main if this is the youngest branch)
[ ] There is a good reason why this PR needs to go into a release branch and this reason is documented (either in the description above or in a linked GitHub issue)

Sep 14 '22 14:09 sfc-gh-mpilman

Result of foundationdb-pr on Linux CentOS 7

Commit ID: d6ace3156ecce896f0ffaccc7fc9df42d2c476c0
Duration 0:05:19
Result: :x: FAILED
Error: Error while executing command: cmake -S . -B build_output -D USE_CCACHE=ON -D USE_WERROR=ON -D RocksDB_ROOT=/opt/rocksdb-6.27.3 -D RUN_JUNIT_TESTS=ON -D RUN_JAVA_INTEGRATION_TESTS=ON -D BUILD_DOCUMENTATION=ON -G Ninja. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

Sep 14 '22 14:09 foundationdb-ci

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: d6ace3156ecce896f0ffaccc7fc9df42d2c476c0
Duration 0:05:17
Result: :x: FAILED
Error: Error while executing command: cmake -S . -B build_output -D USE_CCACHE=ON -D USE_WERROR=ON -D RocksDB_ROOT=/opt/rocksdb-6.27.3 -D RUN_JUNIT_TESTS=OFF -D RUN_JAVA_INTEGRATION_TESTS=OFF -D BUILD_DOCUMENTATION=OFF -G Ninja. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

Sep 14 '22 14:09 foundationdb-ci

Doxense CI Report for Windows 10

Commit ID: d6ace3156ecce896f0ffaccc7fc9df42d2c476c0
Result: :x: FAILED
Build Logs (available for 30 days)

Sep 14 '22 14:09 fdb-windows-ci

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: d555e6253e05a1605dd3e9d05a2e0d4f0aa397cb
Duration 0:04:45
Result: :x: FAILED
Error: Error while executing command: cmake -S . -B build_output -D USE_CCACHE=ON -D USE_WERROR=ON -D RocksDB_ROOT=/opt/rocksdb-6.27.3 -D RUN_JUNIT_TESTS=OFF -D RUN_JAVA_INTEGRATION_TESTS=OFF -D BUILD_DOCUMENTATION=OFF -G Ninja. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

Sep 14 '22 20:09 foundationdb-ci

Result of foundationdb-pr on Linux CentOS 7

Commit ID: d555e6253e05a1605dd3e9d05a2e0d4f0aa397cb
Duration 0:05:10
Result: :x: FAILED
Error: Error while executing command: cmake -S . -B build_output -D USE_CCACHE=ON -D USE_WERROR=ON -D RocksDB_ROOT=/opt/rocksdb-6.27.3 -D RUN_JUNIT_TESTS=ON -D RUN_JAVA_INTEGRATION_TESTS=ON -D BUILD_DOCUMENTATION=ON -G Ninja. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

Sep 14 '22 20:09 foundationdb-ci

Doxense CI Report for Windows 10

Commit ID: d555e6253e05a1605dd3e9d05a2e0d4f0aa397cb
Result: :x: FAILED
Build Logs (available for 30 days)

Sep 14 '22 21:09 fdb-windows-ci

Result of foundationdb-pr-macos on macOS BigSur 11.5.2

Commit ID: 45d5a10e232ff9201be508d53d7afcd35cd70449
Duration 0:05:22
Result: :x: FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /usr/local/bin/bash --login -c ./build_pr_macos.sh. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

Sep 14 '22 22:09 foundationdb-ci

Result of foundationdb-pr on Linux CentOS 7

Commit ID: 45d5a10e232ff9201be508d53d7afcd35cd70449
Duration 0:05:22
Result: :x: FAILED
Error: Error while executing command: cmake -S . -B build_output -D USE_CCACHE=ON -D USE_WERROR=ON -D RocksDB_ROOT=/opt/rocksdb-6.27.3 -D RUN_JUNIT_TESTS=ON -D RUN_JAVA_INTEGRATION_TESTS=ON -D BUILD_DOCUMENTATION=ON -G Ninja. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

Sep 14 '22 22:09 foundationdb-ci

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: 45d5a10e232ff9201be508d53d7afcd35cd70449
Duration 0:05:25
Result: :x: FAILED
Error: Error while executing command: cmake -S . -B build_output -D USE_CCACHE=ON -D USE_WERROR=ON -D RocksDB_ROOT=/opt/rocksdb-6.27.3 -D RUN_JUNIT_TESTS=OFF -D RUN_JAVA_INTEGRATION_TESTS=OFF -D BUILD_DOCUMENTATION=OFF -G Ninja. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

Sep 14 '22 22:09 foundationdb-ci

Doxense CI Report for Windows 10

Commit ID: 45d5a10e232ff9201be508d53d7afcd35cd70449
Result: :x: FAILED
Build Logs (available for 30 days)

Sep 14 '22 22:09 fdb-windows-ci

Result of foundationdb-pr on Linux CentOS 7

Commit ID: 5f62d739bc5f9931c6781971c55de78c6f5da369
Duration 0:05:11
Result: :x: FAILED
Error: Error while executing command: cmake -S . -B build_output -D USE_CCACHE=ON -D USE_WERROR=ON -D RocksDB_ROOT=/opt/rocksdb-6.27.3 -D RUN_JUNIT_TESTS=ON -D RUN_JAVA_INTEGRATION_TESTS=ON -D BUILD_DOCUMENTATION=ON -G Ninja. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

Sep 14 '22 22:09 foundationdb-ci

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: 5f62d739bc5f9931c6781971c55de78c6f5da369
Duration 0:05:17
Result: :x: FAILED
Error: Error while executing command: cmake -S . -B build_output -D USE_CCACHE=ON -D USE_WERROR=ON -D RocksDB_ROOT=/opt/rocksdb-6.27.3 -D RUN_JUNIT_TESTS=OFF -D RUN_JAVA_INTEGRATION_TESTS=OFF -D BUILD_DOCUMENTATION=OFF -G Ninja. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

Sep 14 '22 22:09 foundationdb-ci

Doxense CI Report for Windows 10

Commit ID: 5f62d739bc5f9931c6781971c55de78c6f5da369
Result: :x: FAILED
Build Logs (available for 30 days)

Sep 14 '22 23:09 fdb-windows-ci

The build already takes an unreasonably long time, adding dependency compilation seems like a way to make it take longer.

Sep 15 '22 17:09 ammolitor

This is what caches are for... So the first time it will take a very long time, but we only need to build these dependencies once (or once per configuration -- we use multiple compilers, we might want to compile certain dependencies with sanitizers enabled etc). For CI this will require a change so that the cache survives across runs (AWS codebuild has support for this).

This script generates a hash for each build configuration and it can reuse it. By default it stores this cache in the build directory, but it can be configured to move it somewhere else.

Sep 15 '22 17:09 sfc-gh-mpilman

Result of foundationdb-pr on Linux CentOS 7

Commit ID: 7188298835373d3b6685237e38c83b6f0c7f56ea
Duration 0:05:03
Result: :x: FAILED
Error: Error while executing command: cmake -S . -B build_output -D USE_CCACHE=ON -D USE_WERROR=ON -D RocksDB_ROOT=/opt/rocksdb-6.27.3 -D RUN_JUNIT_TESTS=ON -D RUN_JAVA_INTEGRATION_TESTS=ON -D BUILD_DOCUMENTATION=ON -G Ninja. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

Sep 15 '22 17:09 foundationdb-ci

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: 7188298835373d3b6685237e38c83b6f0c7f56ea
Duration 0:05:40
Result: :x: FAILED
Error: Error while executing command: cmake -S . -B build_output -D USE_CCACHE=ON -D USE_WERROR=ON -D RocksDB_ROOT=/opt/rocksdb-6.27.3 -D RUN_JUNIT_TESTS=OFF -D RUN_JAVA_INTEGRATION_TESTS=OFF -D BUILD_DOCUMENTATION=OFF -G Ninja. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

Sep 15 '22 18:09 foundationdb-ci

Doxense CI Report for Windows 10

Commit ID: 7188298835373d3b6685237e38c83b6f0c7f56ea
Result: :x: FAILED
Build Logs (available for 30 days)

Sep 15 '22 18:09 fdb-windows-ci