cylon
cylon copied to clipboard
Published
20 hours ago •
cylondata
cylondata
build: Build failure debug mode in MacOS
Originally reported by @mstaylor
====================[ Build | cylon | Debug ]===================================
/Applications/CLion.app/Contents/bin/cmake/mac/bin/cmake --build /Users/mstaylor/Documents/phd/code/cylon/cpp/cmake-build-debug --target cylon -j 8
[1/2] Linking CXX shared library lib/libcylon.0.5.0.dylib
FAILED: lib/libcylon.0.5.0.dylib
: && /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -Wall -Wextra -Wno-error=redundant-move -msse4.2 -g -g -fsanitize=address -fno-omit-frame-pointer -arch x86_64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -dynamiclib -Wl,-headerpad_max_install_names -Wl -current_version 0.5.0 -o lib/libcylon.0.5.0.dylib -install_name @rpath/libcylon.0.5.0.dylib src/cylon/CMakeFiles/cylon.dir/arrow/arrow_all_to_all.cpp.o src/cylon/CMakeFiles/cylon.dir/arrow/arrow_buffer.cpp.o src/cylon/CMakeFiles/cylon.dir/arrow/arrow_builder.cpp.o src/cylon/CMakeFiles/cylon.dir/arrow/arrow_comparator.cpp.o src/cylon/CMakeFiles/cylon.dir/arrow/arrow_kernels.cpp.o src/cylon/CMakeFiles/cylon.dir/arrow/arrow_partition_kernels.cpp.o src/cylon/CMakeFiles/cylon.dir/arrow/arrow_task_all_to_all.cpp.o src/cylon/CMakeFiles/cylon.dir/arrow/arrow_types.cpp.o src/cylon/CMakeFiles/cylon.dir/column.cpp.o src/cylon/CMakeFiles/cylon.dir/compute/aggregate_kernels.cpp.o src/cylon/CMakeFiles/cylon.dir/compute/aggregates.cpp.o src/cylon/CMakeFiles/cylon.dir/compute/scalar_aggregate.cpp.o src/cylon/CMakeFiles/cylon.dir/ctx/arrow_memory_pool_utils.cpp.o src/cylon/CMakeFiles/cylon.dir/ctx/cylon_context.cpp.o src/cylon/CMakeFiles/cylon.dir/data_types.cpp.o src/cylon/CMakeFiles/cylon.dir/groupby/groupby.cpp.o src/cylon/CMakeFiles/cylon.dir/groupby/hash_groupby.cpp.o src/cylon/CMakeFiles/cylon.dir/groupby/pipeline_groupby.cpp.o src/cylon/CMakeFiles/cylon.dir/indexing/index.cpp.o src/cylon/CMakeFiles/cylon.dir/indexing/index_utils.cpp.o src/cylon/CMakeFiles/cylon.dir/indexing/indexer.cpp.o src/cylon/CMakeFiles/cylon.dir/indexing/slice.cpp.o src/cylon/CMakeFiles/cylon.dir/io/arrow_io.cpp.o src/cylon/CMakeFiles/cylon.dir/io/csv_read_config.cpp.o src/cylon/CMakeFiles/cylon.dir/io/csv_write_config.cpp.o src/cylon/CMakeFiles/cylon.dir/io/parquet_config.cpp.o src/cylon/CMakeFiles/cylon.dir/join/hash_join.cpp.o src/cylon/CMakeFiles/cylon.dir/join/join.cpp.o src/cylon/CMakeFiles/cylon.dir/join/join_utils.cpp.o src/cylon/CMakeFiles/cylon.dir/join/sort_join.cpp.o src/cylon/CMakeFiles/cylon.dir/mapreduce/mapreduce.cpp.o src/cylon/CMakeFiles/cylon.dir/net/communicator.cpp.o src/cylon/CMakeFiles/cylon.dir/net/mpi/mpi_channel.cpp.o src/cylon/CMakeFiles/cylon.dir/net/mpi/mpi_communicator.cpp.o src/cylon/CMakeFiles/cylon.dir/net/mpi/mpi_operations.cpp.o src/cylon/CMakeFiles/cylon.dir/net/ops/all_to_all.cpp.o src/cylon/CMakeFiles/cylon.dir/net/ops/gather.cpp.o src/cylon/CMakeFiles/cylon.dir/net/ops/bcast.cpp.o src/cylon/CMakeFiles/cylon.dir/net/cylon_request.cpp.o src/cylon/CMakeFiles/cylon.dir/ops.cpp.o src/cylon/CMakeFiles/cylon.dir/ops/all_to_all_op.cpp.o src/cylon/CMakeFiles/cylon.dir/ops/api/parallel_op.cpp.o src/cylon/CMakeFiles/cylon.dir/ops/dis_join_op.cpp.o src/cylon/CMakeFiles/cylon.dir/ops/dis_set_op.cpp.o src/cylon/CMakeFiles/cylon.dir/ops/execution/execution.cpp.o src/cylon/CMakeFiles/cylon.dir/ops/join_op.cpp.o src/cylon/CMakeFiles/cylon.dir/ops/kernels/join_kernel.cpp.o src/cylon/CMakeFiles/cylon.dir/ops/kernels/partition.cpp.o src/cylon/CMakeFiles/cylon.dir/ops/kernels/prepare_array.cpp.o src/cylon/CMakeFiles/cylon.dir/ops/kernels/row_comparator.cpp.o src/cylon/CMakeFiles/cylon.dir/ops/kernels/set_kernel.cpp.o src/cylon/CMakeFiles/cylon.dir/ops/merge_op.cpp.o src/cylon/CMakeFiles/cylon.dir/ops/partition_op.cpp.o src/cylon/CMakeFiles/cylon.dir/ops/set_op.cpp.o src/cylon/CMakeFiles/cylon.dir/ops/split_op.cpp.o src/cylon/CMakeFiles/cylon.dir/partition/partition.cpp.o src/cylon/CMakeFiles/cylon.dir/row.cpp.o src/cylon/CMakeFiles/cylon.dir/serialize/table_serialize.cpp.o src/cylon/CMakeFiles/cylon.dir/table.cpp.o src/cylon/CMakeFiles/cylon.dir/table_api.cpp.o src/cylon/CMakeFiles/cylon.dir/util/arrow_utils.cpp.o src/cylon/CMakeFiles/cylon.dir/util/builtins.cpp.o src/cylon/CMakeFiles/cylon.dir/util/copy_arrray.cpp.o src/cylon/CMakeFiles/cylon.dir/util/flatten_array.cpp.o src/cylon/CMakeFiles/cylon.dir/util/logging.cpp.o src/cylon/CMakeFiles/cylon.dir/util/murmur3.cpp.o src/cylon/CMakeFiles/cylon.dir/util/uuid.cpp.o src/cylon/CMakeFiles/cylon.dir/scalar.cpp.o src/cylon/CMakeFiles/cylon.dir/net/ops/base_ops.cpp.o src/cylon/CMakeFiles/cylon.dir/net/utils.cpp.o -L/Users/mstaylor/opt/anaconda3/envs/cylon_dev/lib -L/Users/mstaylor/opt/anaconda3/envs/cylon_dev/Library/lib -Wl,-rpath,/Users/mstaylor/opt/anaconda3/lib -Wl,-rpath,/usr/local/lib -Wl,-rpath,/Users/mstaylor/opt/anaconda3/envs/cylon_dev/lib -Wl,-rpath,/Users/mstaylor/opt/anaconda3/envs/cylon_dev/Library/lib /Users/mstaylor/opt/anaconda3/lib/libmpicxx.dylib /Users/mstaylor/opt/anaconda3/lib/libmpi.dylib /Users/mstaylor/opt/anaconda3/lib/libpmpi.dylib /usr/local/lib/libglog.0.6.0.dylib /Users/mstaylor/opt/anaconda3/envs/cylon_dev/lib/libarrow.dylib /Users/mstaylor/opt/anaconda3/envs/cylon_dev/lib/libparquet.dylib /usr/local/lib/libgflags.2.2.2.dylib && :
Undefined symbols for architecture x86_64:
“_ompi_mpi_byte”, referenced from:
cylon::MPIChannel::progressReceives() in mpi_channel.cpp.o
cylon::MPIChannel::progressSends() in mpi_channel.cpp.o
cylon::mpi::GetMPIDataType(std::__1::shared_ptr<cylon::DataType> const&) in mpi_operations.cpp.o
“_ompi_mpi_c_bool”, referenced from:
cylon::mpi::GetMPIDataType(std::__1::shared_ptr<cylon::DataType> const&) in mpi_operations.cpp.o
“_ompi_mpi_comm_null”, referenced from:
cylon::net::MPICommunicator::Make(std::__1::shared_ptr<cylon::net::CommConfig> const&, cylon::MemoryPool*, std::__1::shared_ptr<cylon::net::Communicator>*) in mpi_communicator.cpp.o
“_ompi_mpi_comm_world”, referenced from:
cylon::net::MPICommunicator::Make(std::__1::shared_ptr<cylon::net::CommConfig> const&, cylon::MemoryPool*, std::__1::shared_ptr<cylon::net::Communicator>*) in mpi_communicator.cpp.o
“_ompi_mpi_datatype_null”, referenced from:
cylon::mpi::GetMPIDataType(std::__1::shared_ptr<cylon::DataType> const&) in mpi_operations.cpp.o
cylon::mpi::AllReduce(std::__1::shared_ptr<cylon::CylonContext> const&, void const*, void*, int, std::__1::shared_ptr<cylon::DataType> const&, cylon::net::ReduceOp) in mpi_operations.cpp.o
cylon::mpi::MpiAllReduceImpl::AllReduceBuffer(void const*, void*, int, std::__1::shared_ptr<cylon::DataType> const&, cylon::net::ReduceOp) const in mpi_operations.cpp.o
“_ompi_mpi_double”, referenced from:
cylon::mpi::GetMPIDataType(std::__1::shared_ptr<cylon::DataType> const&) in mpi_operations.cpp.o
“_ompi_mpi_errors_return”, referenced from:
cylon::net::MPICommunicator::Make(std::__1::shared_ptr<cylon::net::CommConfig> const&, cylon::MemoryPool*, std::__1::shared_ptr<cylon::net::Communicator>*) in mpi_communicator.cpp.o
“_ompi_mpi_float”, referenced from:
cylon::mpi::GetMPIDataType(std::__1::shared_ptr<cylon::DataType> const&) in mpi_operations.cpp.o
“_ompi_mpi_int”, referenced from:
cylon::MPIChannel::init(int, std::__1::vector<int, std::__1::allocator<int> > const&, std::__1::vector<int, std::__1::allocator<int> > const&, cylon::ChannelReceiveCallback*, cylon::ChannelSendCallback*, cylon::Allocator*) in mpi_channel.cpp.o
cylon::MPIChannel::progressReceives() in mpi_channel.cpp.o
cylon::MPIChannel::sendHeader(std::__1::pair<int const, cylon::PendingSend*> const&) const in mpi_channel.cpp.o
cylon::MPIChannel::sendFinishHeader(std::__1::pair<int const, cylon::PendingSend*> const&) const in mpi_channel.cpp.o
“_ompi_mpi_int16_t”, referenced from:
cylon::mpi::GetMPIDataType(std::__1::shared_ptr<cylon::DataType> const&) in mpi_operations.cpp.o
“_ompi_mpi_int32_t”, referenced from:
cylon::mpi::GetMPIDataType(std::__1::shared_ptr<cylon::DataType> const&) in mpi_operations.cpp.o
cylon::mpi::MpiTableGatherImpl::GatherBufferSizes(int const*, int, int*, int) const in gather.cpp.o
cylon::mpi::GatherArrowBuffer(std::__1::shared_ptr<arrow::Buffer> const&, int, std::__1::shared_ptr<cylon::CylonContext> const&, std::__1::vector<std::__1::shared_ptr<arrow::Buffer>, std::__1::allocator<std::__1::shared_ptr<arrow::Buffer> > >&) in gather.cpp.o
cylon::mpi::MpiTableAllgatherImpl::AllgatherBufferSizes(int const*, int, int*) const in gather.cpp.o
cylon::mpi::AllGatherArrowBuffer(std::__1::shared_ptr<arrow::Buffer> const&, std::__1::shared_ptr<cylon::CylonContext> const&, std::__1::vector<std::__1::shared_ptr<arrow::Buffer>, std::__1::allocator<std::__1::shared_ptr<arrow::Buffer> > >&) in gather.cpp.o
cylon::mpi::MpiAllgatherImpl::AllgatherBufferSize(int const*, int, int*) const in gather.cpp.o
cylon::mpi::MpiTableBcastImpl::BcastBufferSizes(int*, int, int) const in bcast.cpp.o
...
“_ompi_mpi_int64_t”, referenced from:
cylon::mpi::GetMPIDataType(std::__1::shared_ptr<cylon::DataType> const&) in mpi_operations.cpp.o
“_ompi_mpi_int8_t”, referenced from:
cylon::mpi::GetMPIDataType(std::__1::shared_ptr<cylon::DataType> const&) in mpi_operations.cpp.o
“_ompi_mpi_op_band”, referenced from:
cylon::mpi::GetMPIOp(cylon::net::ReduceOp) in mpi_operations.cpp.o
“_ompi_mpi_op_bor”, referenced from:
cylon::mpi::GetMPIOp(cylon::net::ReduceOp) in mpi_operations.cpp.o
“_ompi_mpi_op_land”, referenced from:
cylon::mpi::GetMPIOp(cylon::net::ReduceOp) in mpi_operations.cpp.o
“_ompi_mpi_op_lor”, referenced from:
cylon::mpi::GetMPIOp(cylon::net::ReduceOp) in mpi_operations.cpp.o
“_ompi_mpi_op_max”, referenced from:
cylon::mpi::GetMPIOp(cylon::net::ReduceOp) in mpi_operations.cpp.o
“_ompi_mpi_op_min”, referenced from:
cylon::mpi::GetMPIOp(cylon::net::ReduceOp) in mpi_operations.cpp.o
“_ompi_mpi_op_null”, referenced from:
cylon::mpi::GetMPIOp(cylon::net::ReduceOp) in mpi_operations.cpp.o
cylon::mpi::AllReduce(std::__1::shared_ptr<cylon::CylonContext> const&, void const*, void*, int, std::__1::shared_ptr<cylon::DataType> const&, cylon::net::ReduceOp) in mpi_operations.cpp.o
cylon::mpi::MpiAllReduceImpl::AllReduceBuffer(void const*, void*, int, std::__1::shared_ptr<cylon::DataType> const&, cylon::net::ReduceOp) const in mpi_operations.cpp.o
“_ompi_mpi_op_prod”, referenced from:
cylon::mpi::GetMPIOp(cylon::net::ReduceOp) in mpi_operations.cpp.o
“_ompi_mpi_op_sum”, referenced from:
cylon::mpi::GetMPIOp(cylon::net::ReduceOp) in mpi_operations.cpp.o
“_ompi_mpi_uint16_t”, referenced from:
cylon::mpi::GetMPIDataType(std::__1::shared_ptr<cylon::DataType> const&) in mpi_operations.cpp.o
“_ompi_mpi_uint32_t”, referenced from:
cylon::mpi::GetMPIDataType(std::__1::shared_ptr<cylon::DataType> const&) in mpi_operations.cpp.o
“_ompi_mpi_uint64_t”, referenced from:
…
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.
@mstaylor could you please add the cmake output? Need to see if the MPI libs have been properly picked up
Is this M1or Intel? In other projects I made real bad experience with conda ... I do everything on M1 now from scratch, but that is a different project ...
M1
On Oct 6, 2022, at 12:20 PM, Gregor von Laszewski @.***> wrote:
Is this M1or Intel? In other projects I made real bad experience with conda ... I do everything on M1 now from scratch, but that is a different project ...
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.
I'm able to use the debugger from my Intel MacBook Pro without any of these issues. This flag seems to fix the issue in silicon:
-DCMAKE_OSX_ARCHITECTURES=x86_64
See this: https://intellij-support.jetbrains.com/hc/en-us/community/posts/4406233940242-Using-CLion-on-Apple-Silicon-mac-to-target-x86-64-or-ARM-architecture
Conda install
On Oct 6, 2022, at 4:01 PM, Gregor von Laszewski @.***> wrote:
have you done the conda install or an install from scratch?
— Reply to this email directly, view it on GitHub https://github.com/cylondata/cylon/issues/625#issuecomment-1270612652, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA425C7HG6IKAF4QYYGEQXDWB4VX3ANCNFSM6AAAAAAQ6Y5MJI. You are receiving this because you were mentioned.