Hi there,

This was one of the issues when compiling on macOS. See here: https://forums.foundationdb.org/t/building-on-macos/4491/3

This replaces boost::filesystem with std::filesystem.std::filesystem::recursive_directory_iterator is supported since C++17 and std::string::ends_with is since C++20. FoundationDB is configured to C++20.

Code-Reviewer Section

The general pull request guidelines can be found here.

Please check each of the following things and check all boxes before accepting a PR.

[ ] The PR has a description, explaining both the problem and the solution.
[ ] The description mentions which forms of testing were done and the testing seems reasonable.
[ ] Every function/class/actor that was touched is reasonably well documented.

For Release-Branches

If this PR is made against a release-branch, please also check the following:

[ ] This change/bugfix is a cherry-pick from the next younger branch (younger release-branch or main if this is the youngest branch)
[ ] There is a good reason why this PR needs to go into a release branch and this reason is documented (either in the description above or in a linked GitHub issue)

Aug 09 '24 10:08 sepeth

@xis19 Since you were discussing about using std::filesystem, it'd be good if you can have a look.

Aug 09 '24 17:08 spraza

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Commit ID: d72ecd4a9b7b7fac1e2515c98d5ba66b8d7358ba
Duration 0:06:22
Result: :x: FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /opt/homebrew/bin/bash --login -c ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

Aug 09 '24 20:08 foundationdb-ci

Result of foundationdb-pr-macos on macOS Ventura 13.x

Commit ID: d72ecd4a9b7b7fac1e2515c98d5ba66b8d7358ba
Duration 0:11:08
Result: :x: FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /usr/local/bin/bash --login -c ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

Aug 09 '24 20:08 foundationdb-ci

Result of foundationdb-pr-clang-ide on Linux CentOS 7

Commit ID: d72ecd4a9b7b7fac1e2515c98d5ba66b8d7358ba
Duration 0:26:03
Result: :white_check_mark: SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

Aug 09 '24 21:08 foundationdb-ci

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: d72ecd4a9b7b7fac1e2515c98d5ba66b8d7358ba
Duration 0:26:38
Result: :x: FAILED
Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

Aug 09 '24 21:08 foundationdb-ci

Result of foundationdb-pr on Linux CentOS 7

Commit ID: d72ecd4a9b7b7fac1e2515c98d5ba66b8d7358ba
Duration 0:29:37
Result: :x: FAILED
Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

Aug 09 '24 21:08 foundationdb-ci

Result of foundationdb-pr-clang-arm on Linux CentOS 7

Commit ID: d72ecd4a9b7b7fac1e2515c98d5ba66b8d7358ba
Duration 0:51:46
Result: :white_check_mark: SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

Aug 09 '24 21:08 foundationdb-ci

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: d72ecd4a9b7b7fac1e2515c98d5ba66b8d7358ba
Duration 1:15:36
Result: :white_check_mark: SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

Aug 09 '24 22:08 foundationdb-ci

Seems to be related to this https://github.com/apple/foundationdb/pull/11564

Currently trying to use std::filesystem over what we have currently. This is still a long-effort since there's a lot to refactor through. Leaving this here as a note

Aug 15 '24 14:08 ghost

Seems to be related to this #11564

Currently trying to use std::filesystem over what we have currently. This is still a long-effort since there's a lot to refactor through. Leaving this here as a note

Awesomeness ^-^ I remember seeing your change, I wanted to do this separately though, as it is tiny in scope, and solves something painful for me on macOS. I had to keep stashing/commenting this bit to be able to compile on macOS.

Aug 15 '24 17:08 sepeth

Hmm, looks like foundationdb-pr check fails with:

fdbserver/tester.actor.cpp: In function 'void encryptionAtRestPlaintextMarkerCheck()':
fdbserver/tester.actor.cpp:2558:6: error: 'path' is not a member of 'fs'
fdbserver/tester.actor.cpp:2559:6: error: 'recursive_directory_iterator' is not a member of 'fs'
fdbserver/tester.actor.cpp:2563:11: error: 'recursive_directory_iterator' is not a member of 'fs'
fdbserver/tester.actor.cpp:2563:48: error: 'itr' was not declared in this scope
fdbserver/tester.actor.cpp:2563:55: error: 'end' was not declared in this scope

Is it ok to use something like the following here?

#ifdef __APPLE__ // or clang
using fs = std::filesystem;
#else
using fs = boost::filesystem;
#end

Aug 22 '24 14:08 sepeth

From https://libcxx.llvm.org/Status/Cxx17.html#note-p0156 it seems filesystem is fully supported in clang libc++ since version 7.0. Can you post full compiler error? Really hate to compile FDB using a Mac laptop, takes forever

Aug 22 '24 17:08 xis19

From https://libcxx.llvm.org/Status/Cxx17.html#note-p0156 it seems filesystem is fully supported in clang libc++ since version 7.0. Can you post full compiler error? Really hate to compile FDB using a Mac laptop, takes forever

Oh, the last error I shared above is from GCC on Linux from the checks above. See this foundationdb-pr check log: https://github.com/apple/foundationdb/pull/11566#issuecomment-2278765966

Clang supports std::filesystem, but linking boost::filesystem, which the current code uses, fails on macOS for some reason. I mentioned this in the forum a while ago here: https://forums.foundationdb.org/t/building-on-macos/4491/3

For that reason, I thought I can switch this code to use std::filesystem, but now GCC is not happy.

But this is still curious, as it should be supported by GCC 11 / libstdc++ as well if I am reading this correctly: https://en.cppreference.com/w/cpp/compiler_support/17 and this: https://gcc.gnu.org/onlinedocs/libstdc++/manual/status.html#status.iso.2017

Aug 22 '24 18:08 sepeth

Oh 🙈 I think the problem is that I forgot to #include <filesystem>.

Aug 22 '24 18:08 sepeth

May you please kick off PR checks again? I believe they will pass this time ^-^

Aug 23 '24 19:08 sepeth

Hi @spraza and @xis19, ICYMI ^-^

May you please kick off PR checks again? I believe they will pass this time ^-^

Sep 17 '24 16:09 sepeth

Hi @spraza and @xis19, ICYMI ^-^

May you please kick off PR checks again? I believe they will pass this time ^-^

@sepeth LGTM. Triggered CI. Once it passes, will accept and merge the PR.

Sep 17 '24 18:09 spraza

Result of foundationdb-pr-clang-ide on Linux CentOS 7

Commit ID: b614ba764bc4257ae0592bb3bb3f4823a22c2d77
Duration 0:22:21
Result: :white_check_mark: SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

Sep 17 '24 18:09 foundationdb-ci

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Commit ID: b614ba764bc4257ae0592bb3bb3f4823a22c2d77
Duration 0:42:42
Result: :white_check_mark: SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

Sep 17 '24 18:09 foundationdb-ci