arrow icon indicating copy to clipboard operation
arrow copied to clipboard
verified publisher icon apache

[GLib][Ruby] undefined method 'compute_initialize' for module Arrow (NoMethodError)

Open hiroyuki-sato opened this issue 6 months ago • 8 comments

Describe the bug, including details regarding any error messages, version, and platform.

Raise the exception below when ARROW_COMPUTE OFF

( cd c_glib ; ../c_glib/test/run-test.sh  )
/path/to/arrow/c_glib/test/run-test.rb:34:in '<main>': undefined method 'compute_initialize' for module Arrow (NoMethodError)

Arrow.compute_initialize
     ^^^^^^^^^^^^^^^^^^^

Reproduce steps

git clone https://github.com/apache/arrow/
cd arrow/cpp
mkdir build
cd build
cmake .. --preset ninja-debug-maximal \
  -DCMAKE_INSTALL_PREFIX=/tmp/local \
  -DARROW_CUDA=OFF \
  -DARROW_SKYHOOK=OFF \
  -DCMAKE_OSX_SYSROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk \
  -DARROW_EXTRA_ERROR_CONTEXT=OFF
cmake --build .
cmake --install .

meson setup \
      --backend=ninja \
      --prefix=/tmp/local \
      --libdir=lib \
      --pkg-config-path=/tmp/local/lib/pkgconfig \
      -Ddoc=false \
      -Dvapi=false \
      -Dwerror=false \
      c_glib.build \
      c_glib
ninja -C c_glib.build all

Environment

  • macOS: 15.5
  • Apache Arrow: 5f0aeb5de53fb25b59a52661a80071faef99a4a4

Component(s)

GLib

hiroyuki-sato avatar Jun 20 '25 16:06 hiroyuki-sato

It's strange because ninja-debug-maximal must have ARROW_COMPUTE=ON.

kou avatar Jun 20 '25 20:06 kou

Not sure if this is related, but I noticed the module’s pkg-config file doesn’t include the compute CFLAGS lines:

Cflags: @ARROW_COMPUTE_PC_CFLAGS@
Cflags.private: @ARROW_COMPUTE_PC_CFLAGS_PRIVATE@

Without those, the Compute component’s headers/symbols might not get exposed even when ARROW_COMPUTE is ON. Could that be why Arrow.compute_initialize isn’t defined?

  • https://github.com/apache/arrow/blob/main/cpp/src/arrow/arrow-compute.pc.in
  • https://github.com/apache/arrow/blob/main/cpp/src/arrow/compute/arrow-compute.pc.in

otegami avatar Jun 21 '25 04:06 otegami

Thanks. I was something wrong. It resolved after re-install with latest source (a9d2a12f6135b5595b71ec267dbff33a114a0b5e).

However..

Another problem occurred. If this is a new problem, I'll create another issue.

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x20)
    frame #0: 0x0000000000000020
error: memory read failed for 0x20
Target 0: (ruby) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x20)
  * frame #0: 0x0000000000000020
    frame #1: 0x000000011f05127c libprotobuf.29.3.0.dylib`google::protobuf::internal::TcParser::AddMessage(google::protobuf::internal::TcParseTableBase const*, google::protobuf::internal::RepeatedPtrFieldBase&) + 124
    frame #2: 0x000000011f052d28 libprotobuf.29.3.0.dylib`google::protobuf::internal::TcParser::FastMtR1(google::protobuf::MessageLite*, char const*, google::protobuf::internal::ParseContext*, google::protobuf::internal::TcFieldData, google::protobuf::internal::TcParseTableBase const*, unsigned long long) + 100
    frame #3: 0x000000011f0a5c94 libprotobuf.29.3.0.dylib`bool google::protobuf::internal::MergeFromImpl<false>(google::protobuf::io::ZeroCopyInputStream*, google::protobuf::MessageLite*, google::protobuf::internal::TcParseTableBase const*, google::protobuf::MessageLite::ParseFlags) + 188
    frame #4: 0x000000011f0a6dd8 libprotobuf.29.3.0.dylib`google::protobuf::MessageLite::ParseFromZeroCopyStream(google::protobuf::io::ZeroCopyInputStream*) + 24
    frame #5: 0x000000012ec18f68 libarrow.2100.0.0.dylib`orc::readFooter(stream=0x0000600002108b20, buffer=0x0000600002922d30, footerOffset=1265, ps=0x0000600000117240, memoryPool=0x000000012f0769e8, readerMetrics=0x0000000000000000) at Reader.cc:1353:18
    frame #6: 0x000000012ec19928 libarrow.2100.0.0.dylib`orc::createReader(stream=orc::InputStream @ 0x0000600002108b20, options=0x000000016fdfb3e8) at Reader.cc:1411:26
    frame #7: 0x000000012e3ec174 libarrow.2100.0.0.dylib`arrow::adapters::orc::ORCFileReader::Impl::Open(this=0x0000600002922d00, file=std::__1::shared_ptr<arrow::io::RandomAccessFile>::element_type @ 0x00006000001a0f60 strong=3 weak=2, pool=0x000000012f06ea40) at adapter.cc:218:5
    frame #8: 0x000000012e3ebf54 libarrow.2100.0.0.dylib`arrow::adapters::orc::ORCFileReader::Open(file=std::__1::shared_ptr<arrow::io::RandomAccessFile>::element_type @ 0x00006000001a0f60 strong=3 weak=2, pool=0x000000012f06ea40) at adapter.cc:568:3
    frame #9: 0x000000011cc63a7c libarrow-glib.2100.dylib`garrow_orc_file_reader_new(input=0x0000600003f34260, error=0x000000016fdfb6f0) at orc-file-reader.cpp:173:5
    frame #10: 0x000000019e7e0050 libffi.dylib`ffi_call_SYSV + 80
    frame #11: 0x000000019e7e8af0 libffi.dylib`ffi_call_int + 1220
    frame #12: 0x000000011b99dbf4 libgirepository-1.0.1.dylib`g_callable_info_invoke + 860
    frame #13: 0x000000011b99ef28 libgirepository-1.0.1.dylib`g_function_info_invoke + 252
    frame #14: 0x0000000100776068 gobject_introspection.bundle`rb_gi_function_info_invoke_raw + 212
    frame #15: 0x0000000100774e48 gobject_introspection.bundle`rg_invoke + 72
    frame #16: 0x0000000100a5ce2c libruby.3.4.dylib`vm_call_cfunc_with_frame_ + 240
    frame #17: 0x0000000100a40848 libruby.3.4.dylib`vm_exec_core + 7860
    frame #18: 0x0000000100a3d47c libruby.3.4.dylib`rb_vm_exec + 492
    frame #19: 0x0000000100a64dac libruby.3.4.dylib`invoke_block_from_c_bh + 872
    frame #20: 0x0000000100a644ec libruby.3.4.dylib`catch_i + 108
    frame #21: 0x0000000100a4f01c libruby.3.4.dylib`vm_catch_protect + 200
    frame #22: 0x0000000100a4fa2c libruby.3.4.dylib`rb_f_catch + 112
    frame #23: 0x0000000100a5ce2c libruby.3.4.dylib`vm_call_cfunc_with_frame_ + 240
    frame #24: 0x0000000100a40848 libruby.3.4.dylib`vm_exec_core + 7860
    frame #25: 0x0000000100a3d47c libruby.3.4.dylib`rb_vm_exec + 492
    frame #26: 0x0000000100a64dac libruby.3.4.dylib`invoke_block_from_c_bh + 872
    frame #27: 0x0000000100a4ce08 libruby.3.4.dylib`rb_yield + 168
    frame #28: 0x00000001007f3d7c libruby.3.4.dylib`rb_ary_each + 64
    frame #29: 0x0000000100a5ce2c libruby.3.4.dylib`vm_call_cfunc_with_frame_ + 240
    frame #30: 0x0000000100a40848 libruby.3.4.dylib`vm_exec_core + 7860
    frame #31: 0x0000000100a3d47c libruby.3.4.dylib`rb_vm_exec + 492
    frame #32: 0x0000000100a636e8 libruby.3.4.dylib`rb_call0 + 888
    frame #33: 0x00000001009372e4 libruby.3.4.dylib`rb_class_new_instance_pass_kw + 64
    frame #34: 0x0000000100a5ce2c libruby.3.4.dylib`vm_call_cfunc_with_frame_ + 240
    frame #35: 0x0000000100a40848 libruby.3.4.dylib`vm_exec_core + 7860
    frame #36: 0x0000000100a3d47c libruby.3.4.dylib`rb_vm_exec + 492
    frame #37: 0x0000000100a64dac libruby.3.4.dylib`invoke_block_from_c_bh + 872
    frame #38: 0x0000000100a644ec libruby.3.4.dylib`catch_i + 108
    frame #39: 0x0000000100a4f01c libruby.3.4.dylib`vm_catch_protect + 200
    frame #40: 0x0000000100a4fa2c libruby.3.4.dylib`rb_f_catch + 112
    frame #41: 0x0000000100a5ce2c libruby.3.4.dylib`vm_call_cfunc_with_frame_ + 240
    frame #42: 0x0000000100a40848 libruby.3.4.dylib`vm_exec_core + 7860
    frame #43: 0x0000000100a3d47c libruby.3.4.dylib`rb_vm_exec + 492
    frame #44: 0x0000000100a64dac libruby.3.4.dylib`invoke_block_from_c_bh + 872
    frame #45: 0x0000000100a4ce08 libruby.3.4.dylib`rb_yield + 168
    frame #46: 0x00000001007f3d7c libruby.3.4.dylib`rb_ary_each + 64
    frame #47: 0x0000000100a5ce2c libruby.3.4.dylib`vm_call_cfunc_with_frame_ + 240
    frame #48: 0x0000000100a40848 libruby.3.4.dylib`vm_exec_core + 7860
    frame #49: 0x0000000100a3d47c libruby.3.4.dylib`rb_vm_exec + 492
    frame #50: 0x0000000100a64dac libruby.3.4.dylib`invoke_block_from_c_bh + 872
    frame #51: 0x0000000100a4ce08 libruby.3.4.dylib`rb_yield + 168
    frame #52: 0x00000001007f3d7c libruby.3.4.dylib`rb_ary_each + 64
    frame #53: 0x0000000100a5ce2c libruby.3.4.dylib`vm_call_cfunc_with_frame_ + 240
    frame #54: 0x0000000100a40848 libruby.3.4.dylib`vm_exec_core + 7860
    frame #55: 0x0000000100a3d47c libruby.3.4.dylib`rb_vm_exec + 492
    frame #56: 0x0000000100a64dac libruby.3.4.dylib`invoke_block_from_c_bh + 872
    frame #57: 0x0000000100a644ec libruby.3.4.dylib`catch_i + 108
    frame #58: 0x0000000100a4f01c libruby.3.4.dylib`vm_catch_protect + 200
    frame #59: 0x0000000100a4fa2c libruby.3.4.dylib`rb_f_catch + 112
    frame #60: 0x0000000100a5ce2c libruby.3.4.dylib`vm_call_cfunc_with_frame_ + 240
    frame #61: 0x0000000100a40848 libruby.3.4.dylib`vm_exec_core + 7860
    frame #62: 0x0000000100a3d664 libruby.3.4.dylib`rb_vm_exec + 980
    frame #63: 0x00000001008ad7c0 libruby.3.4.dylib`rb_ec_exec_node + 152
    frame #64: 0x00000001008ad6e0 libruby.3.4.dylib`ruby_run_node + 68
    frame #65: 0x0000000100003f24 ruby`main + 104
    frame #66: 0x000000018bcb6b98 dyld`start + 6076

hiroyuki-sato avatar Jun 21 '25 05:06 hiroyuki-sato

Could you try it again after rm -rf /tmp/local?

Multiple ProtoBuf instances may be mixed.

kou avatar Jun 21 '25 07:06 kou

Still crashed. I removed /tmp/local and executed ccache --clear, then rebuild cpp and c_glib

grep protobuf /tmp/log2.txt
/opt/homebrew/Cellar/protobuf/29.3/lib/libprotobuf.29.3.0.dylib(_ZN6google8protobuf8internal8TcParser10AddMessageEPKNS1_16TcParseTableBaseERNS1_20RepeatedPtrFieldBaseE+0x7c) [0x12249127c]
/opt/homebrew/Cellar/protobuf/29.3/lib/libprotobuf.29.3.0.dylib(_ZN6google8protobuf8internal8TcParser8FastMtR1EPNS0_11MessageLiteEPKcPNS1_12ParseContextENS1_11TcFieldDataEPKNS1_16TcParseTableBaseEy) [0x122492d28]
/opt/homebrew/Cellar/protobuf/29.3/lib/libprotobuf.29.3.0.dylib(_ZN6google8protobuf8internal13MergeFromImplILb0EEEbPNS0_2io19ZeroCopyInputStreamEPNS0_11MessageLiteEPKNS1_16TcParseTableBaseENS6_10ParseFlagsE) [0x1224e5c94]
/opt/homebrew/Cellar/protobuf/29.3/lib/libprotobuf.29.3.0.dylib(_ZN6google8protobuf11MessageLite23ParseFromZeroCopyStreamEPNS0_2io19ZeroCopyInputStreamE) [0x1224e6dd8]
102e70000-102e80000 rw- /opt/homebrew/Cellar/protobuf/29.3/lib/libutf8_validity.dylib
102e80000-102e84000 r-x /opt/homebrew/Cellar/protobuf/29.3/lib/libutf8_validity.dylib
102e84000-102e8c000 r-- /opt/homebrew/Cellar/protobuf/29.3/lib/libutf8_validity.dylib
1223d8000-12254c000 r-x /opt/homebrew/Cellar/protobuf/29.3/lib/libprotobuf.29.3.0.dylib
12254c000-122554000 r-- /opt/homebrew/Cellar/protobuf/29.3/lib/libprotobuf.29.3.0.dylib
122554000-12255c000 rw- /opt/homebrew/Cellar/protobuf/29.3/lib/libprotobuf.29.3.0.dylib
12255c000-1226b0000 r-- /opt/homebrew/Cellar/protobuf/29.3/lib/libprotobuf.29.3.0.dylib
140ec8000-1411a0000 r-- /opt/homebrew/Cellar/protobuf/29.3/lib/libprotobuf.29.3.0.dylib

log.txt log2.txt

hiroyuki-sato avatar Jun 21 '25 07:06 hiroyuki-sato

I also attached cpp build log.

log3.txt

hiroyuki-sato avatar Jun 21 '25 07:06 hiroyuki-sato

I couldn't review this in detail yet, but could the issue described here be related? https://github.com/protocolbuffers/protobuf/issues/21447

otegami avatar Jun 22 '25 01:06 otegami

@otegami Thanks I would not know whether protobuf build with gcc or clang yet. I believe Arrow build with clang.

hiroyuki-sato avatar Jun 23 '25 02:06 hiroyuki-sato