Builds with Clang on macOS <= 10.13 fail: `Undefined symbols for architecture x86_64: "std::__1::__fs::filesystem::directory_iterator::__dereference() const"`

[barracuda156]

Builds with Clang on macOS <= 10.13 fail at linking:

[100%] Linking CXX shared library ../lib/libpika.dylib
cd /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_devel_pika/pika/work/build/libs && /opt/local/bin/cmake -E cmake_link_script CMakeFiles/pika.dir/link.txt --verbose=ON
/opt/local/bin/mpicxx-mpich-clang15 -pipe -I/opt/local/libexec/boost/1.81/include -Os -DNDEBUG -I/opt/local/libexec/boost/1.81/include -I/opt/local/include -stdlib=libc++ -arch x86_64 -mmacosx-version-min=10.13 -dynamiclib -Wl,-headerpad_max_install_names -pthread -L/opt/local/libexec/boost/1.81/lib -L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-rpath,/opt/local/lib/libgcc -current_version 0.12.0 -o ../lib/libpika.0.12.0.dylib -install_name @rpath/libpika.0.dylib CMakeFiles/pika.dir/src/dummy.cpp.o pika/affinity/CMakeFiles/pika_affinity.dir/Unity/unity_0_cxx.cxx.o pika/assertion/CMakeFiles/pika_assertion.dir/Unity/unity_0_cxx.cxx.o pika/async_base/CMakeFiles/pika_async_base.dir/Unity/unity_0_cxx.cxx.o pika/async_mpi/CMakeFiles/pika_async_mpi.dir/Unity/unity_0_cxx.cxx.o pika/command_line_handling/CMakeFiles/pika_command_line_handling.dir/Unity/unity_0_cxx.cxx.o pika/concurrency/CMakeFiles/pika_concurrency.dir/Unity/unity_0_cxx.cxx.o pika/config/CMakeFiles/pika_config.dir/Unity/unity_0_cxx.cxx.o pika/coroutines/CMakeFiles/pika_coroutines.dir/Unity/unity_0_cxx.cxx.o pika/coroutines/CMakeFiles/pika_coroutines.dir/src/detail/coroutine_impl.cpp.o pika/coroutines/CMakeFiles/pika_coroutines.dir/src/detail/coroutine_self.cpp.o pika/debugging/CMakeFiles/pika_debugging.dir/Unity/unity_0_cxx.cxx.o pika/errors/CMakeFiles/pika_errors.dir/Unity/unity_0_cxx.cxx.o pika/execution/CMakeFiles/pika_execution.dir/Unity/unity_0_cxx.cxx.o pika/execution_base/CMakeFiles/pika_execution_base.dir/Unity/unity_0_cxx.cxx.o pika/executors/CMakeFiles/pika_executors.dir/Unity/unity_0_cxx.cxx.o pika/filesystem/CMakeFiles/pika_filesystem.dir/Unity/unity_0_cxx.cxx.o pika/functional/CMakeFiles/pika_functional.dir/Unity/unity_0_cxx.cxx.o pika/futures/CMakeFiles/pika_futures.dir/Unity/unity_0_cxx.cxx.o pika/ini/CMakeFiles/pika_ini.dir/Unity/unity_0_cxx.cxx.o pika/init_runtime/CMakeFiles/pika_init_runtime.dir/Unity/unity_0_cxx.cxx.o pika/itt_notify/CMakeFiles/pika_itt_notify.dir/Unity/unity_0_cxx.cxx.o pika/lcos/CMakeFiles/pika_lcos.dir/Unity/unity_0_cxx.cxx.o pika/lock_registration/CMakeFiles/pika_lock_registration.dir/Unity/unity_0_cxx.cxx.o pika/logging/CMakeFiles/pika_logging.dir/Unity/unity_1_cxx.cxx.o pika/logging/CMakeFiles/pika_logging.dir/Unity/unity_0_cxx.cxx.o pika/mpi_base/CMakeFiles/pika_mpi_base.dir/Unity/unity_0_cxx.cxx.o pika/prefix/CMakeFiles/pika_prefix.dir/Unity/unity_0_cxx.cxx.o pika/program_options/CMakeFiles/pika_program_options.dir/Unity/unity_1_cxx.cxx.o pika/program_options/CMakeFiles/pika_program_options.dir/Unity/unity_0_cxx.cxx.o pika/resource_partitioner/CMakeFiles/pika_resource_partitioner.dir/Unity/unity_0_cxx.cxx.o pika/runtime_configuration/CMakeFiles/pika_runtime_configuration.dir/Unity/unity_0_cxx.cxx.o pika/runtime/CMakeFiles/pika_runtime.dir/Unity/unity_1_cxx.cxx.o pika/runtime/CMakeFiles/pika_runtime.dir/Unity/unity_0_cxx.cxx.o pika/schedulers/CMakeFiles/pika_schedulers.dir/Unity/unity_0_cxx.cxx.o pika/string_util/CMakeFiles/pika_string_util.dir/Unity/unity_0_cxx.cxx.o pika/synchronization/CMakeFiles/pika_synchronization.dir/Unity/unity_0_cxx.cxx.o pika/thread_pool_util/CMakeFiles/pika_thread_pool_util.dir/Unity/unity_0_cxx.cxx.o pika/thread_pools/CMakeFiles/pika_thread_pools.dir/Unity/unity_0_cxx.cxx.o pika/thread_support/CMakeFiles/pika_thread_support.dir/Unity/unity_0_cxx.cxx.o pika/threading/CMakeFiles/pika_threading.dir/Unity/unity_0_cxx.cxx.o pika/threading_base/CMakeFiles/pika_threading_base.dir/Unity/unity_2_cxx.cxx.o pika/threading_base/CMakeFiles/pika_threading_base.dir/Unity/unity_1_cxx.cxx.o pika/threading_base/CMakeFiles/pika_threading_base.dir/Unity/unity_0_cxx.cxx.o pika/thread_manager/CMakeFiles/pika_thread_manager.dir/Unity/unity_0_cxx.cxx.o pika/topology/CMakeFiles/pika_topology.dir/Unity/unity_0_cxx.cxx.o pika/util/CMakeFiles/pika_util.dir/Unity/unity_0_cxx.cxx.o pika/version/CMakeFiles/pika_version.dir/Unity/unity_0_cxx.cxx.o  -Wl,-rpath,/opt/local/lib /opt/local/lib/libhwloc.dylib /opt/local/lib/libjemalloc.dylib /opt/local/libexec/boost/1.81/lib/libboost_context-mt.dylib -ldl /opt/local/lib/libfmt9/libfmt.9.0.0.dylib 
Undefined symbols for architecture x86_64:
  "std::__1::__fs::filesystem::directory_iterator::__dereference() const", referenced from:
      pika::util::merge_component_inis(pika::util::section&) in unity_0_cxx.cxx.o
  "std::__1::__fs::filesystem::path::__filename() const", referenced from:
      __ZNKSt3__14__fs10filesystem4path8filenameB6v15007Ev in unity_0_cxx.cxx.o
      __ZNSt3__14__fs10filesystem4pathdVB6v15007ERKS2_ in unity_0_cxx.cxx.o
      __ZNSt3__14__fs10filesystem4pathdVB6v15007ERKS2_ in unity_0_cxx.cxx.o
      __ZNSt3__14__fs10filesystem4pathdVB6v15007ERKS2_ in unity_0_cxx.cxx.o
  "std::__1::__fs::filesystem::path::__extension() const", referenced from:
      __ZNKSt3__14__fs10filesystem4path9extensionB6v15007Ev in unity_0_cxx.cxx.o
  "std::__1::__fs::filesystem::path::__parent_path() const", referenced from:
      __ZNKSt3__14__fs10filesystem4path11parent_pathB6v15007Ev in unity_0_cxx.cxx.o
      __ZNKSt3__14__fs10filesystem4path11parent_pathB6v15007Ev in unity_0_cxx.cxx.o
  "std::__1::__fs::filesystem::path::__root_directory() const", referenced from:
      __ZNSt3__14__fs10filesystem4pathdVB6v15007ERKS2_ in unity_0_cxx.cxx.o
      pika::detail::filesystem::canonical(std::__1::__fs::filesystem::path const&, std::__1::__fs::filesystem::path const&) in unity_0_cxx.cxx.o
      pika::detail::filesystem::canonical(std::__1::__fs::filesystem::path const&, std::__1::__fs::filesystem::path const&, std::__1::error_code&) in unity_0_cxx.cxx.o
      __ZNSt3__14__fs10filesystem4pathdVB6v15007ERKS2_ in unity_0_cxx.cxx.o
      __ZNSt3__14__fs10filesystem4pathdVB6v15007ERKS2_ in unity_0_cxx.cxx.o
  "std::__1::__fs::filesystem::path::__stem() const", referenced from:
      __ZNKSt3__14__fs10filesystem4path4stemB6v15007Ev in unity_0_cxx.cxx.o
  "std::__1::__fs::filesystem::path::__compare(std::__1::basic_string_view<char, std::__1::char_traits<char> >) const", referenced from:
      __ZNSt3__14__fs10filesystemeqB6v15007ERKNS1_4pathES4_ in unity_0_cxx.cxx.o
      __ZNSt3__14__fs10filesystemneB6v15007ERKNS1_4pathES4_ in unity_0_cxx.cxx.o
  "std::__1::__fs::filesystem::__canonical(std::__1::__fs::filesystem::path const&, std::__1::error_code*)", referenced from:
      pika::detail::filesystem::canonical(std::__1::__fs::filesystem::path const&, std::__1::__fs::filesystem::path const&) in unity_0_cxx.cxx.o
      pika::detail::filesystem::canonical(std::__1::__fs::filesystem::path const&, std::__1::__fs::filesystem::path const&, std::__1::error_code&) in unity_0_cxx.cxx.o
  "std::__1::__fs::filesystem::__current_path(std::__1::error_code*)", referenced from:
      pika::detail::filesystem::initial_path() in unity_0_cxx.cxx.o
      pika::util::init_ini_data_base(pika::util::section&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&) in unity_0_cxx.cxx.o
  "std::__1::__fs::filesystem::directory_iterator::__increment(std::__1::error_code*)", referenced from:
      pika::util::merge_component_inis(pika::util::section&) in unity_0_cxx.cxx.o
  "std::__1::__fs::filesystem::directory_iterator::directory_iterator(std::__1::__fs::filesystem::path const&, std::__1::error_code*, std::__1::__fs::filesystem::directory_options)", referenced from:
      pika::util::merge_component_inis(pika::util::section&) in unity_0_cxx.cxx.o
  "std::__1::__fs::filesystem::__status(std::__1::__fs::filesystem::path const&, std::__1::error_code*)", referenced from:
      pika::util::handle_ini_file(pika::util::section&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) in unity_0_cxx.cxx.o
      pika::util::init_ini_data_base(pika::util::section&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&) in unity_0_cxx.cxx.o
      pika::util::merge_component_inis(pika::util::section&) in unity_0_cxx.cxx.o
  "typeinfo for std::__1::__fs::filesystem::filesystem_error", referenced from:
      GCC_except_table14 in unity_0_cxx.cxx.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [lib/libpika.0.12.0.dylib] Error 1

Logs: https://build.macports.org/builders/ports-10.13_x86_64-builder/builds/179626/steps/install-port/logs/stdio https://build.macports.org/builders/ports-10.12_x86_64-builder/builds/217679/steps/install-port/logs/stdio

[msimberg]

I think this is wontfix from our side. We unconditionally require support for std::filesystem and macOS was very late with adding proper support for it (meaning without additional flags or libraries; the first version where no flags or libraries are required is 10.15). That said, I think you may be able to get this to work by linking to -lc++fs (either in CXXFLAGS or CMAKE_CXX_FLAGS). I would prefer not to add this to our build system.

[barracuda156]

@msimberg Hmm, but pika builds fine with GCC on 10.6.8, that too for ppc in Rosetta.

P. S. I did not run the complete test suite on 10.6.8, but 100% of header tests passed, and there are no errors during the build itself.

[msimberg]

Ok, I'm quite sure what to think of that... macOS is a bit funny? I do know that GCC (specifically, libstdc++) got support for std::filesystem without linking to stdc++fs in version 9, so that's sort of consistent. I also know that Apple's clang is a special snowflake (and it looks like it's using libc++), but I don't use macOS myself so I don't really know much more than that. Would you mind trying linking to c++fs?

[barracuda156]

@msimberg I have made a PR, once merged, we will see if it helped (I have no environment locally to test Clang builds). https://github.com/macports/macports-ports/pull/17740

[msimberg]

@barracuda156 all right, thanks for the update and for updating the recipe! Let's see :crossed_fingers:

[msimberg]

I'll close this with the assumption that this has been fixed. @barracuda156 please feel free to reopen if you think this is still an issue!