bazel-buildfarm icon indicating copy to clipboard operation
bazel-buildfarm copied to clipboard

Published 20 hours ago • verified publisher icon bazelbuild

Running a server on Linux and build on MacOS

Open physxP opened this issue 1 year ago • 3 comments

I followed the server setup guide on my linux machine. I also setup a worker and redis db there exactly as mentioned in the readme file.

Now if I try to run the example main.cc in the QuickStart Docs, from my macbook m1, I get the following error:

❯ bazel run --remote_executor=grpc://192.168.18.125:8980 :main
INFO: Invocation ID: 2ae0c841-c3f0-40f2-9734-cab7d8cef75a
INFO: Analyzed target //:main (0 packages loaded, 0 targets configured).
ERROR: /Users/tj/projects/buildfarm-test/BUILD:1:10: Compiling main.cc failed: (Exit 1): cc_wrapper.sh failed: error executing CppCompile command (from target //:main) external/bazel_tools~cc_configure_extension~local_config_cc/cc_wrapper.sh -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -Wunused-but-set-parameter -Wno-free-nonheap-object ... (remaining 25 arguments skipped)
gcc: error: unrecognized command-line option '-Wthread-safety'
gcc: error: unrecognized command-line option '-Wself-assign'
gcc: error: unrecognized command-line option '-fcolor-diagnostics'
gcc: error: unrecognized command-line option '-mmacosx-version-min=14.4'
Target //:main failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.240s, Critical Path: 0.13s
INFO: 2 processes: 2 internal.
ERROR: Build did NOT complete successfully
ERROR: Build failed. Not running target

It builds fine locally and from another linux machine:

tj@pop-os:~/projects/buildfarm-test$ bazel run --remote_executor=grpc://192.168.18.125:8980 :main
2024/05/14 01:18:48 Downloading https://releases.bazel.build/7.1.2/release/bazel-7.1.2-linux-x86_64...
Downloading: 66 MB out of 66 MB (100%) 
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Invocation ID: 4cab047c-349d-46ff-ad4e-3e11382e23bc
INFO: Analyzed target //:main (67 packages loaded, 302 targets configured).
INFO: Found 1 target...
Target //:main up-to-date:
  bazel-bin/main
INFO: Elapsed time: 11.836s, Critical Path: 0.67s
INFO: 7 processes: 5 internal, 2 remote.
INFO: Build completed successfully, 7 total actions
INFO: Running command line: bazel-bin/main
Hello, World!

physxP avatar May 13 '24 20:05 physxP

I checked the QuickStart on my Macbook (M2 Pro), the latest macOS. It seems to work fine...

% bazel run --remote_executor=grpc://localhost:8980 :main
INFO: Invocation ID: 30b04d0d-23b3-46d3-ae7f-8a8d40f371ec
INFO: Analyzed target //:main (68 packages loaded, 348 targets configured).
INFO: Found 1 target...
Target //:main up-to-date:
  bazel-bin/main
INFO: Elapsed time: 1.399s, Critical Path: 1.12s
INFO: 7 processes: 5 internal, 2 remote.
INFO: Build completed successfully, 7 total actions
INFO: Running command line: bazel-bin/main
Hello, World!

% gcc --version
Apple clang version 15.0.0 (clang-1500.3.9.4)
Target: arm64-apple-darwin23.4.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

monaka avatar May 14 '24 06:05 monaka

Hi @monaka, thanks for the quick check. The problem I am facing is that I am running the buildfarm on my linux machine over my network (not localhost). When I try to run the simple hello world from my macbook I get the issue. If I run it using another linux machine I don't have this problem.

What I gathered from this was that since on my mac I have clang and remote machine has gcc, clang flags are somehow passed to remote gcc. If I force clang on my remote machine then it comes up with another error:

❯ bazel run --repo_env=CC=clang --toolchain_resolution_debug='@bazel_tools//tools/cpp:toolchain_type' --remote_executor=grpc://192.168.18.125:8980 :main
INFO: Invocation ID: d6fb945a-d353-415b-a361-cc016f57c602
WARNING: Build option --toolchain_resolution_debug has changed, discarding analysis cache (this can be expensive, see https://bazel.build/advanced/performance/iteration-speed).
INFO: ToolchainResolution: Performing resolution of @@bazel_tools//tools/cpp:toolchain_type for target platform @@local_config_platform//:host
      ToolchainResolution:   Rejected toolchain @@bazel_tools~cc_configure_extension~local_config_cc//:cc-compiler-armeabi-v7a; mismatching values: armv7, android
      ToolchainResolution:   Toolchain @@bazel_tools~cc_configure_extension~local_config_cc//:cc-compiler-darwin_arm64 is compatible with target plaform, searching for execution platforms:
      ToolchainResolution:     Compatible execution platform @@local_config_platform//:host
      ToolchainResolution:   All execution platforms have been assigned a @@bazel_tools//tools/cpp:toolchain_type toolchain, stopping
      ToolchainResolution: Recap of selected @@bazel_tools//tools/cpp:toolchain_type toolchains for target platform @@local_config_platform//:host:
      ToolchainResolution:   Selected @@bazel_tools~cc_configure_extension~local_config_cc//:cc-compiler-darwin_arm64 to run on execution platform @@local_config_platform//:host
INFO: ToolchainResolution: Target platform @@local_config_platform//:host: Selected execution platform @@local_config_platform//:host, type @@bazel_tools//tools/cpp:toolchain_type -> toolchain @@bazel_tools~cc_configure_extension~local_config_cc//:cc-compiler-darwin_arm64
INFO: Analyzed target //:main (0 packages loaded, 355 targets configured).
ERROR: /Users/tj/projects/buildfarm-test/BUILD:3:10: Compiling main.cc failed: absolute path inclusion(s) found in rule '//:main':
the source file 'main.cc' includes the following non-builtin files with absolute paths (if these are builtin files, make sure these paths are in your toolchain):
  '/usr/include/c++/12/iostream'
  '/usr/include/x86_64-linux-gnu/c++/12/bits/c++config.h'
  '/usr/include/x86_64-linux-gnu/c++/12/bits/os_defines.h'
  '/usr/include/features.h'
  '/usr/include/features-time64.h'
  '/usr/include/x86_64-linux-gnu/bits/wordsize.h'
  '/usr/include/x86_64-linux-gnu/bits/timesize.h'
  '/usr/include/stdc-predef.h'
  '/usr/include/x86_64-linux-gnu/sys/cdefs.h'
  '/usr/include/x86_64-linux-gnu/bits/long-double.h'
  '/usr/include/x86_64-linux-gnu/gnu/stubs.h'
  '/usr/include/x86_64-linux-gnu/gnu/stubs-64.h'
  '/usr/include/x86_64-linux-gnu/c++/12/bits/cpu_defines.h'
  '/usr/include/c++/12/ostream'
  '/usr/include/c++/12/ios'
  '/usr/include/c++/12/iosfwd'
  '/usr/include/c++/12/bits/stringfwd.h'
  '/usr/include/c++/12/bits/memoryfwd.h'
  '/usr/include/c++/12/bits/postypes.h'
  '/usr/include/c++/12/cwchar'
  '/usr/include/wchar.h'
  '/usr/include/x86_64-linux-gnu/bits/libc-header-start.h'
  '/usr/include/x86_64-linux-gnu/bits/floatn.h'
  '/usr/include/x86_64-linux-gnu/bits/floatn-common.h'
  '/usr/lib/clang/14.0.0/include/stddef.h'
  '/usr/lib/clang/14.0.0/include/stdarg.h'
  '/usr/include/x86_64-linux-gnu/bits/wchar.h'
  '/usr/include/x86_64-linux-gnu/bits/types/wint_t.h'
  '/usr/include/x86_64-linux-gnu/bits/types/mbstate_t.h'
  '/usr/include/x86_64-linux-gnu/bits/types/__mbstate_t.h'
  '/usr/include/x86_64-linux-gnu/bits/types/__FILE.h'
  '/usr/include/x86_64-linux-gnu/bits/types/FILE.h'
  '/usr/include/x86_64-linux-gnu/bits/types/locale_t.h'
  '/usr/include/x86_64-linux-gnu/bits/types/__locale_t.h'
  '/usr/include/c++/12/exception'
  '/usr/include/c++/12/bits/exception.h'
  '/usr/include/c++/12/bits/exception_ptr.h'
  '/usr/include/c++/12/bits/exception_defines.h'
  '/usr/include/c++/12/bits/cxxabi_init_exception.h'
  '/usr/lib/clang/14.0.0/include/__stddef_max_align_t.h'
  '/usr/include/c++/12/typeinfo'
  '/usr/include/c++/12/bits/hash_bytes.h'
  '/usr/include/c++/12/new'
  '/usr/include/c++/12/bits/move.h'
  '/usr/include/c++/12/type_traits'
  '/usr/include/c++/12/bits/nested_exception.h'
  '/usr/include/c++/12/bits/char_traits.h'
  '/usr/include/c++/12/cstdint'
  '/usr/lib/clang/14.0.0/include/stdint.h'
  '/usr/include/stdint.h'
  '/usr/include/x86_64-linux-gnu/bits/types.h'
  '/usr/include/x86_64-linux-gnu/bits/typesizes.h'
  '/usr/include/x86_64-linux-gnu/bits/time64.h'
  '/usr/include/x86_64-linux-gnu/bits/stdint-intn.h'
  '/usr/include/x86_64-linux-gnu/bits/stdint-uintn.h'
  '/usr/include/c++/12/bits/localefwd.h'
  '/usr/include/x86_64-linux-gnu/c++/12/bits/c++locale.h'
  '/usr/include/c++/12/clocale'
  '/usr/include/locale.h'
  '/usr/include/x86_64-linux-gnu/bits/locale.h'
  '/usr/include/c++/12/cctype'
  '/usr/include/ctype.h'
  '/usr/include/x86_64-linux-gnu/bits/endian.h'
  '/usr/include/x86_64-linux-gnu/bits/endianness.h'
  '/usr/include/c++/12/bits/ios_base.h'
  '/usr/include/c++/12/ext/atomicity.h'
  '/usr/include/x86_64-linux-gnu/c++/12/bits/gthr.h'
  '/usr/include/x86_64-linux-gnu/c++/12/bits/gthr-default.h'
  '/usr/include/pthread.h'
  '/usr/include/sched.h'
  '/usr/include/x86_64-linux-gnu/bits/types/time_t.h'
  '/usr/include/x86_64-linux-gnu/bits/types/struct_timespec.h'
  '/usr/include/x86_64-linux-gnu/bits/sched.h'
  '/usr/include/x86_64-linux-gnu/bits/types/struct_sched_param.h'
  '/usr/include/x86_64-linux-gnu/bits/cpu-set.h'
  '/usr/include/time.h'
  '/usr/include/x86_64-linux-gnu/bits/time.h'
  '/usr/include/x86_64-linux-gnu/bits/timex.h'
  '/usr/include/x86_64-linux-gnu/bits/types/struct_timeval.h'
  '/usr/include/x86_64-linux-gnu/bits/types/clock_t.h'
  '/usr/include/x86_64-linux-gnu/bits/types/struct_tm.h'
  '/usr/include/x86_64-linux-gnu/bits/types/clockid_t.h'
  '/usr/include/x86_64-linux-gnu/bits/types/timer_t.h'
  '/usr/include/x86_64-linux-gnu/bits/types/struct_itimerspec.h'
  '/usr/include/x86_64-linux-gnu/bits/pthreadtypes.h'
  '/usr/include/x86_64-linux-gnu/bits/thread-shared-types.h'
  '/usr/include/x86_64-linux-gnu/bits/pthreadtypes-arch.h'
  '/usr/include/x86_64-linux-gnu/bits/atomic_wide_counter.h'
  '/usr/include/x86_64-linux-gnu/bits/struct_mutex.h'
  '/usr/include/x86_64-linux-gnu/bits/struct_rwlock.h'
  '/usr/include/x86_64-linux-gnu/bits/setjmp.h'
  '/usr/include/x86_64-linux-gnu/bits/types/__sigset_t.h'
  '/usr/include/x86_64-linux-gnu/bits/types/struct___jmp_buf_tag.h'
  '/usr/include/x86_64-linux-gnu/bits/pthread_stack_min-dynamic.h'
  '/usr/include/x86_64-linux-gnu/c++/12/bits/atomic_word.h'
  '/usr/include/x86_64-linux-gnu/sys/single_threaded.h'
  '/usr/include/c++/12/bits/locale_classes.h'
  '/usr/include/c++/12/string'
  '/usr/include/c++/12/bits/allocator.h'
  '/usr/include/x86_64-linux-gnu/c++/12/bits/c++allocator.h'
  '/usr/include/c++/12/bits/new_allocator.h'
  '/usr/include/c++/12/bits/functexcept.h'
  '/usr/include/c++/12/bits/cpp_type_traits.h'
  '/usr/include/c++/12/bits/ostream_insert.h'
  '/usr/include/c++/12/bits/cxxabi_forced.h'
  '/usr/include/c++/12/bits/stl_iterator_base_types.h'
  '/usr/include/c++/12/bits/stl_iterator_base_funcs.h'
  '/usr/include/c++/12/bits/concept_check.h'
  '/usr/include/c++/12/debug/assertions.h'
  '/usr/include/c++/12/bits/stl_iterator.h'
  '/usr/include/c++/12/ext/type_traits.h'
  '/usr/include/c++/12/bits/ptr_traits.h'
  '/usr/include/c++/12/bits/stl_function.h'
  '/usr/include/c++/12/backward/binders.h'
  '/usr/include/c++/12/ext/numeric_traits.h'
  '/usr/include/c++/12/bits/stl_algobase.h'
  '/usr/include/c++/12/bits/stl_pair.h'
  '/usr/include/c++/12/bits/utility.h'
  '/usr/include/c++/12/debug/debug.h'
  '/usr/include/c++/12/bits/predefined_ops.h'
  '/usr/include/c++/12/bits/refwrap.h'
  '/usr/include/c++/12/bits/invoke.h'
  '/usr/include/c++/12/bits/range_access.h'
  '/usr/include/c++/12/initializer_list'
  '/usr/include/c++/12/bits/basic_string.h'
  '/usr/include/c++/12/ext/alloc_traits.h'
  '/usr/include/c++/12/bits/alloc_traits.h'
  '/usr/include/c++/12/bits/stl_construct.h'
  '/usr/include/c++/12/ext/string_conversions.h'
  '/usr/include/c++/12/cstdlib'
  '/usr/include/stdlib.h'
  '/usr/include/x86_64-linux-gnu/bits/waitflags.h'
  '/usr/include/x86_64-linux-gnu/bits/waitstatus.h'
  '/usr/include/x86_64-linux-gnu/sys/types.h'
  '/usr/include/endian.h'
  '/usr/include/x86_64-linux-gnu/bits/byteswap.h'
  '/usr/include/x86_64-linux-gnu/bits/uintn-identity.h'
  '/usr/include/x86_64-linux-gnu/sys/select.h'
  '/usr/include/x86_64-linux-gnu/bits/select.h'
  '/usr/include/x86_64-linux-gnu/bits/types/sigset_t.h'
  '/usr/include/alloca.h'
  '/usr/include/x86_64-linux-gnu/bits/stdlib-float.h'
  '/usr/include/c++/12/bits/std_abs.h'
  '/usr/include/c++/12/cstdio'
  '/usr/include/stdio.h'
  '/usr/include/x86_64-linux-gnu/bits/types/__fpos_t.h'
  '/usr/include/x86_64-linux-gnu/bits/types/__fpos64_t.h'
  '/usr/include/x86_64-linux-gnu/bits/types/struct_FILE.h'
  '/usr/include/x86_64-linux-gnu/bits/types/cookie_io_functions_t.h'
  '/usr/include/x86_64-linux-gnu/bits/stdio_lim.h'
  '/usr/include/c++/12/cerrno'
  '/usr/include/errno.h'
  '/usr/include/x86_64-linux-gnu/bits/errno.h'
  '/usr/include/linux/errno.h'
  '/usr/include/x86_64-linux-gnu/asm/errno.h'
  '/usr/include/asm-generic/errno.h'
  '/usr/include/asm-generic/errno-base.h'
  '/usr/include/x86_64-linux-gnu/bits/types/error_t.h'
  '/usr/include/c++/12/bits/charconv.h'
  '/usr/include/c++/12/bits/functional_hash.h'
  '/usr/include/c++/12/bits/basic_string.tcc'
  '/usr/include/c++/12/bits/locale_classes.tcc'
  '/usr/include/c++/12/system_error'
  '/usr/include/x86_64-linux-gnu/c++/12/bits/error_constants.h'
  '/usr/include/c++/12/stdexcept'
  '/usr/include/c++/12/streambuf'
  '/usr/include/c++/12/bits/streambuf.tcc'
  '/usr/include/c++/12/bits/basic_ios.h'
  '/usr/include/c++/12/bits/locale_facets.h'
  '/usr/include/c++/12/cwctype'
  '/usr/include/wctype.h'
  '/usr/include/x86_64-linux-gnu/bits/wctype-wchar.h'
  '/usr/include/x86_64-linux-gnu/c++/12/bits/ctype_base.h'
  '/usr/include/c++/12/bits/streambuf_iterator.h'
  '/usr/include/x86_64-linux-gnu/c++/12/bits/ctype_inline.h'
  '/usr/include/c++/12/bits/locale_facets.tcc'
  '/usr/include/c++/12/bits/basic_ios.tcc'
  '/usr/include/c++/12/bits/ostream.tcc'
  '/usr/include/c++/12/istream'
  '/usr/include/c++/12/bits/istream.tcc'
clang: warning: argument unused during compilation: '-mmacosx-version-min=14.4' [-Wunused-command-line-argument]
Target //:main failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.243s, Critical Path: 0.03s
INFO: 2 processes: 2 internal.
ERROR: Build did NOT complete successfully
ERROR: Build failed. Not running target

physxP avatar May 14 '24 08:05 physxP

The difficulty here will be in qualifying an 'exec' platform (distinct from 'host' and 'target') for your buildfarm executions. https://bazel.build/concepts/platforms has a rough description of this, but not from the remote perspective. exec_compatible_with should be the mechanism applied to your toolchain executables and configuration.

I have not arrived at a recipe with a working example for the situation you're describing, where:

  • The 'host' platform is a macos machine, running bazel
  • The 'target' platform is something the resultant binaries or libraries will run or load on, respectively
  • The 'exec' platform is linux, with a toolchain that can build for the 'target' platform
  • The 'host' client knows that the remote is 'exec' based, or how to decorate its actions' platforms to for execution

I don't know what the 'exec' platform is in your example, are you building for linux, or for macos? If you have any luck with exec_compatible_with please post back here. Otherwise I can explore some of this scenario, but I'm not confident I'll have a working example soon.

werkt avatar May 21 '24 12:05 werkt