truffleruby icon indicating copy to clipboard operation
truffleruby copied to clipboard

Gem commonmarker 1.x fails to compile native extensions with rust

Open rwstauner opened this issue 1 year ago • 14 comments

Using the latest commit and a jvm-ce build

truffleruby 24.1.0-dev-00307c24, like ruby 3.2.2, GraalVM CE JVM [x86_64-linux]

with the latest gem version (1.0.4) installation fails due to several items missing from RbConfig::CONFIG:

error: failed to run custom build command for `rb-sys v0.9.85`

Caused by:
  process didn't exit successfully: `/home/spin/.gem/truffleruby/3.2.2/gems/commonmarker-1.0.4/ext/commonmarker/target/release/build/rb-sys-4475e29d236d2359/build-script-main` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=RUBY
  cargo:rerun-if-env-changed=RBCONFIG_CROSS_COMPILING
  cargo:rerun-if-env-changed=RBCONFIG_RUBY_PROGRAM_VERSION
  cargo:rerun-if-env-changed=RBCONFIG_MAJOR

  --- stderr
  thread 'main' panicked at /home/spin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rb-sys-build-0.9.85/src/rb_config.rs:206:32:
  Key not found: MAJOR

If I add these values (and use something like export TRUFFLERUBYOPT=-r$PWD/config) I get past those issues.

RbConfig::CONFIG["MAJOR"] = "3"
RbConfig::CONFIG["MINOR"] = "2"
RbConfig::CONFIG["TEENY"] = "1"
RbConfig::CONFIG["PATCHLEVEL"] = "1"
RbConfig::CONFIG["archincludedir"] = "/tmp/archincludedir"

Then I get other errors:

error[E0432]: unresolved import `crate::rb_gc_adjust_memory_usage`
 --> /home/spin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rb-sys-0.9.85/src/tracking_allocator.rs:3:13
  |
3 | use crate::{rb_gc_adjust_memory_usage, utils::is_ruby_vm_started};
  |             ^^^^^^^^^^^^^^^^^^^^^^^^^ no `rb_gc_adjust_memory_usage` in the root

error[E0412]: cannot find type `RData` in this scope
-->
/home/spin/.gem/truffleruby/3.2.2/gems/commonmarker-1.0.4/ext/commonmarker/target/release/build/rb-sys-f73ca150f0763414/out/bindings-0.9.85-x86_64-linux-3.2.1.rs:3478:51
     |
3478 |         pub fn rb_tr_rdata(object: VALUE) -> *mut RData;
     |                                                   ^^^^^ not found in this scope

error[E0609]: no field `flags` on type `uncategorized::RBasic`
  --> /home/spin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rb-sys-0.9.85/src/stable_api/ruby_3_2.rs:19:35
   |
19 |         let flags = rstring.basic.flags;
   |                                   ^^^^^ unknown field

error[E0609]: no field `flags` on type `uncategorized::RBasic`
  --> /home/spin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rb-sys-0.9.85/src/stable_api/ruby_3_2.rs:34:35
   |
34 |         let flags = rstring.basic.flags;
   |                                   ^^^^^ unknown field

error[E0609]: no field `flags` on type `uncategorized::RBasic`
  --> /home/spin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rb-sys-0.9.85/src/stable_api/ruby_3_2.rs:52:34
   |
52 |         let flags = rarray.basic.flags;
   |                                  ^^^^^ unknown field

error[E0609]: no field `flags` on type `uncategorized::RBasic`
  --> /home/spin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rb-sys-0.9.85/src/stable_api/ruby_3_2.rs:56:38
   |
56 |             let mut f = rarray.basic.flags;
   |                                      ^^^^^ unknown field

error[E0609]: no field `flags` on type `uncategorized::RBasic`
  --> /home/spin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rb-sys-0.9.85/src/stable_api/ruby_3_2.rs:70:34
   |
70 |         let flags = rarray.basic.flags;
   |                                  ^^^^^ unknown field

error[E0609]: no field `flags` on type `uncategorized::RBasic`
  --> /home/spin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rb-sys-0.9.85/src/stable_api/ruby_3_2.rs:94:35
   |
94 |         let ret: u32 = ((*rbasic).flags & crate::ruby_value_type::RUBY_T_MASK as VALUE) as _;
   |                                   ^^^^^ unknown field

Some errors have detailed explanations: E0412, E0432, E0609.
For more information about an error, try `rustc --explain E0412`.
error: could not compile `rb-sys` (lib) due to 8 previous errors
warning: build failed, waiting for other jobs to finish...
make: *** [Makefile:377: target/release/libcommonmarker.so] Error 101

make failed, exit code 2

rwstauner avatar Jan 18 '24 23:01 rwstauner

Gems that use the rb-sys for rust don't appear to work. I'm running into an identical issue with truffleruby 24.0.2 trying to install the polars-df gem (https://github.com/ankane/ruby-polars) which also uses the rb-sys crate.

jac241 avatar Sep 02 '24 05:09 jac241

I have heard @nirvdrum has been looking at this recently. In general TruffleRuby should be able to support Rust extensions easier now that extensions are run natively. However there seems to be some issues like the missing RbConfig entries above and rb-sys accessing too many internals that need to be addressed first.

eregon avatar Sep 12 '24 15:09 eregon

With https://github.com/oracle/truffleruby/pull/3664 fixed, I tried gem i -V commonmarker and got:

error: failed to run custom build command for `rb-sys v0.9.99`
...
  thread 'main' panicked at /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-1.1.5/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.69.4/lib.rs:622:31:
  Unable to find libclang: "couldn't find any valid shared libraries matching: ['libclang.so', 'libclang-*.so', 'libclang.so.*', 'libclang-*.so.*'], set the `LIBCLANG_PATH` environment variable to a path where one of these files can be found (invalid: [])"

OK I guess I need to install libclang on Linux: https://github.com/oxidize-rb/rb-sys?tab=readme-ov-file#what-dependencies-do-i-need-to-build-a-ruby-extension-in-rust After sudo dnf install clang-libs:

  /usr/include/stdio.h:33:10: fatal error: 'stddef.h' file not found
  thread 'main' panicked at /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-1.1.5/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/rb-sys-0.9.99/build/main.rs:55:6:
  generate bindings: ClangDiagnostic("/usr/include/stdio.h:33:10: fatal error: 'stddef.h' file not found\n")

OK, so after sudo dnf install clang:

  error: 'rustfmt' is not installed for the toolchain 'stable-x86_64-unknown-linux-gnu'.
  To install, run `rustup component add rustfmt`
  Failed to run rustfmt: Internal rustfmt error (non-fatal, continuing)
  error: 'rustfmt' is not installed for the toolchain 'stable-x86_64-unknown-linux-gnu'.
  To install, run `rustup component add rustfmt`
  thread 'main' panicked at /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-1.1.5/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/rb-sys-build-0.9.99/src/rb_config.rs:209:32:
  Key not found: archincludedir
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

So we are still missing RbConfig::CONFIG['archincludedir'] it seems, we should define that like MRI does:

> RbConfig::MAKEFILE_CONFIG['archincludedir']
=> "$(includedir)/$(arch)"

^ @andrykonchin Could you make a PR adding that?

eregon avatar Sep 17 '24 16:09 eregon

Doing a quick workaround for that, similar to what Randy did in the first post:

# config.rb
RbConfig::CONFIG["archincludedir"] = "#{RbConfig::CONFIG["includedir"]}/#{RbConfig::CONFIG["arch"]}"

I get:

error[E0432]: unresolved import `crate::rb_gc_adjust_memory_usage`
 --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-1.1.5/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/rb-sys-0.9.99/src/tracking_allocator.rs:3:13
  |
3 | use crate::{rb_gc_adjust_memory_usage, utils::is_ruby_vm_started};
  |             ^^^^^^^^^^^^^^^^^^^^^^^^^ no `rb_gc_adjust_memory_usage` in the root

error[E0412]: cannot find type `RData` in this scope
 --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-1.1.5/ext/commonmarker/target/release/build/rb-sys-b6e62330b9516279/out/bindings-0.9.99-x86_64-linux-3.2.4.rs:1:299289
  |
1 | ....2.4"] pub fn rb_tr_rdata (object : VALUE) -> * mut RData ; # [doc = "This is the primitive way to wrap an existing C struct into ::RD...
  |                                                        ^^^^^ not found in this scope

   Compiling emojis v0.6.3
error[E0609]: no field `flags` on type `uncategorized::RBasic`
  --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-1.1.5/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/rb-sys-0.9.99/src/stable_api/ruby_3_2.rs:19:35
   |
19 |         let flags = rstring.basic.flags;
   |                                   ^^^^^ unknown field

error[E0609]: no field `flags` on type `uncategorized::RBasic`
  --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-1.1.5/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/rb-sys-0.9.99/src/stable_api/ruby_3_2.rs:34:35
   |
34 |         let flags = rstring.basic.flags;
   |                                   ^^^^^ unknown field

error[E0609]: no field `flags` on type `uncategorized::RBasic`
  --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-1.1.5/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/rb-sys-0.9.99/src/stable_api/ruby_3_2.rs:52:34
   |
52 |         let flags = rarray.basic.flags;
   |                                  ^^^^^ unknown field

error[E0609]: no field `flags` on type `uncategorized::RBasic`
  --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-1.1.5/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/rb-sys-0.9.99/src/stable_api/ruby_3_2.rs:56:38
   |
56 |             let mut f = rarray.basic.flags;
   |                                      ^^^^^ unknown field

error[E0609]: no field `flags` on type `uncategorized::RBasic`
  --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-1.1.5/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/rb-sys-0.9.99/src/stable_api/ruby_3_2.rs:70:34
   |
70 |         let flags = rarray.basic.flags;
   |                                  ^^^^^ unknown field

error[E0609]: no field `flags` on type `uncategorized::RBasic`
  --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-1.1.5/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/rb-sys-0.9.99/src/stable_api/ruby_3_2.rs:94:35
   |
94 |         let ret: u32 = ((*rbasic).flags & crate::ruby_value_type::RUBY_T_MASK as VALUE) as _;
   |                                   ^^^^^ unknown field

Some errors have detailed explanations: E0412, E0432, E0609.
For more information about an error, try `rustc --explain E0412`.
error: could not compile `rb-sys` (lib) due to 8 previous errors

Re rb_gc_adjust_memory_usage, TruffleRuby does define that as static inline: https://github.com/oracle/truffleruby/blob/e834bb5fe3d0877f5a664901adb655fd86b6b63c/lib/cext/include/ruby/internal/intern/gc.h#L388-L395 I guess Rust doesn't find it because it doesn't end up in the libruby then, we could fix that by declaring it in src/main/c/cext/gc.c instead.

Regarding RData, no clue.

Regarding RBasic.flags that's something that TruffleRuby on purpose does not and cannot expose, so that's something that will need to be fixed in rb-sys so it doesn't assume struct RBasic to have those fields.

eregon avatar Sep 17 '24 16:09 eregon

@andrykonchin added archincludedir and moved rb_gc_adjust_memory_usage in https://github.com/oracle/truffleruby/pull/3671. So now the remaining issues are in the rb-sys crate.

eregon avatar Sep 23 '24 11:09 eregon

Looking at the first flags error, it comes from https://github.com/oxidize-rb/rb-sys/blob/991df6de21d244631e1b889070a8e4136f4220fe/crates/rb-sys/src/stable_api/ruby_3_2.rs#L19 rb-sys is hardcoding CRuby internals, but it should use the RSTRING_LEN macro instead.

eregon avatar Sep 23 '24 11:09 eregon

I found some more info in https://github.com/oxidize-rb/rb-sys/pull/229. So probably TruffleRuby should be treated like ruby-head then, because ruby_3_2.rs hardcoded macros are incompatible and CRuby-specific.

eregon avatar Sep 23 '24 11:09 eregon

I submitted an issue to rb-sys: https://github.com/oxidize-rb/rb-sys/issues/424

eregon avatar Sep 23 '24 11:09 eregon

Would be good to try again now that https://github.com/oxidize-rb/rb-sys/issues/424 got fixed

eregon avatar Nov 19 '24 15:11 eregon

On truffleruby-head: truffleruby 25.0.0-dev-c4775937, like ruby 3.3.5, GraalVM CE Native [x86_64-linux]

$ gem i -V commonmarker
...
Fetching rb_sys-0.9.106.gem
Fetching commonmarker-2.0.2.1.gem
...
error[E0432]: unresolved imports `rb_sys::rb_bug`, `rb_sys::rb_warning`
 --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-2.0.2.1/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/magnus-0.7.1/src/error.rs:8:5
  |
8 |     rb_bug, rb_ensure, rb_errinfo, rb_exc_raise, rb_iter_break_value, rb_jump_tag, rb_protect,
  |     ^^^^^^ no `rb_bug` in the root
9 |     rb_set_errinfo, rb_warning, ruby_special_consts, VALUE,
  |                     ^^^^^^^^^^
  |                     |
  |                     no `rb_warning` in the root
  |                     help: a similar name exists in the module: `rb_String`

error[E0432]: unresolved import `rb_sys::rb_struct_define`
  --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-2.0.2.1/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/magnus-0.7.1/src/r_struct.rs:17:37
   |
17 |     rb_struct_aref, rb_struct_aset, rb_struct_define, rb_struct_getmember, rb_struct_members,
   |                                     ^^^^^^^^^^^^^^^^
   |                                     |
   |                                     no `rb_struct_define` in the root
   |                                     help: a similar name exists in the module: `rb_struct_size`

error[E0432]: unresolved import `rb_sys::rb_thread_wait_for`
 --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-2.0.2.1/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/magnus-0.7.1/src/thread.rs:8:5
  |
8 |     rb_thread_wait_for, rb_thread_wakeup, rb_thread_wakeup_alive, timeval, VALUE,
  |     ^^^^^^^^^^^^^^^^^^
  |     |
  |     no `rb_thread_wait_for` in the root
  |     help: a similar name exists in the module: `rb_thread_wait_fd`

error[E0433]: failed to resolve: could not find `ruby_rvalue_flags` in `rb_sys`
  --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-2.0.2.1/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/magnus-0.7.1/src/r_struct.rs:45:44
   |
45 |     pub const EMBED_LEN_MAX: u32 = rb_sys::ruby_rvalue_flags::RVALUE_EMBED_LEN_MAX as u32;
   |                                            ^^^^^^^^^^^^^^^^^
   |                                            |
   |                                            could not find `ruby_rvalue_flags` in `rb_sys`
   |                                            help: an enum with a similar name exists: `ruby_rmodule_flags`

error[E0308]: mismatched types
   --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-2.0.2.1/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/magnus-0.7.1/src/error.rs:564:24
    |
564 | pub fn bug(s: &str) -> ! {
    |        ---             ^ expected `!`, found `()`
    |        |
    |        implicitly returns `()` as its body has no tail or `return` expression
    |
    = note:   expected type `!`
            found unit type `()`

error[E0609]: no field `flags` on type `&rb_sys::RBasic`
   --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-2.0.2.1/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/magnus-0.7.1/src/r_bignum.rs:426:30
    |
426 |             r_basic.as_ref().flags & (ruby_fl_type::RUBY_FL_USER1 as VALUE) != 0
    |                              ^^^^^ unknown field

error[E0609]: no field `flags` on type `&rb_sys::RBasic`
   --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-2.0.2.1/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/magnus-0.7.1/src/r_string.rs:914:52
    |
914 |                 (self.r_basic_unchecked().as_ref().flags
    |                                                    ^^^^^ unknown field

error[E0599]: no variant or associated item named `RUBY_ENC_CODERANGE_MASK` found for enum `ruby_coderange_type` in the current scope
   --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-2.0.2.1/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/magnus-0.7.1/src/r_string.rs:915:44
    |
915 |                     & ruby_coderange_type::RUBY_ENC_CODERANGE_MASK as VALUE) as u32,
    |                                            ^^^^^^^^^^^^^^^^^^^^^^^ variant or associated item not found in `ruby_coderange_type`
    |
help: there is a variant with a similar name
    |
915 |                     & ruby_coderange_type::RUBY_ENC_CODERANGE_7BIT as VALUE) as u32,
    |                                            ~~~~~~~~~~~~~~~~~~~~~~~

error[E0609]: no field `flags` on type `&mut rb_sys::RBasic`
   --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-2.0.2.1/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/magnus-0.7.1/src/r_string.rs:964:47
    |
964 |             self.r_basic_unchecked().as_mut().flags &=
    |                                               ^^^^^ unknown field

error[E0599]: no variant or associated item named `RUBY_ENC_CODERANGE_MASK` found for enum `ruby_coderange_type` in the current scope
   --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-2.0.2.1/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/magnus-0.7.1/src/r_string.rs:965:40
    |
965 |                 !(ruby_coderange_type::RUBY_ENC_CODERANGE_MASK as VALUE)
    |                                        ^^^^^^^^^^^^^^^^^^^^^^^ variant or associated item not found in `ruby_coderange_type`
    |
help: there is a variant with a similar name
    |
965 |                 !(ruby_coderange_type::RUBY_ENC_CODERANGE_7BIT as VALUE)
    |                                        ~~~~~~~~~~~~~~~~~~~~~~~

error[E0609]: no field `flags` on type `&mut rb_sys::RBasic`
    --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-2.0.2.1/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/magnus-0.7.1/src/r_string.rs:1022:43
     |
1022 |         self.r_basic_unchecked().as_mut().flags |= cr as VALUE;
     |                                           ^^^^^ unknown field

error[E0609]: no field `flags` on type `&rb_sys::RBasic`
    --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-2.0.2.1/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/magnus-0.7.1/src/r_string.rs:1302:47
     |
1302 |             self.r_basic_unchecked().as_ref().flags & ruby_rstring_flags::RSTRING_FSTR as VALUE != 0
     |                                               ^^^^^ unknown field

error[E0609]: no field `flags` on type `&rb_sys::RBasic`
   --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-2.0.2.1/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/magnus-0.7.1/src/r_struct.rs:131:38
    |
131 |         let flags = r_basic.as_ref().flags;
    |                                      ^^^^^ unknown field

error[E0609]: no field `flags` on type `&rb_sys::RBasic`
   --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-2.0.2.1/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/magnus-0.7.1/src/value.rs:723:52
    |
723 |                         let ret = r_basic.as_ref().flags & (ruby_value_type::RUBY_T_MASK as VALUE);
    |                                                    ^^^^^ unknown field

error[E0609]: no field `klass` on type `&rb_sys::RBasic`
   --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-2.0.2.1/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/magnus-0.7.1/src/value.rs:983:83
    |
983 |                 Some(r_basic) => RClass::from_rb_value_unchecked(r_basic.as_ref().klass),
    |                                                                                   ^^^^^ unknown field

error[E0609]: no field `flags` on type `&rb_sys::RBasic`
    --> /home/eregon/.rubies/truffleruby-dev/lib/gems/gems/commonmarker-2.0.2.1/ext/commonmarker/.rb-sys/stable/cargo/registry/src/index.crates.io-6f17d22bba15001f/magnus-0.7.1/src/value.rs:1029:34
     |
1029 |                 r_basic.as_ref().flags & ruby_fl_type::RUBY_FL_FREEZE as VALUE != 0
     |                                  ^^^^^ unknown field

Some functions like rb_struct_define() look missing to Rust but actually TruffleRuby defines them, but as static inline, e.g. https://github.com/oracle/truffleruby/blob/c4775937ca5d89adb010584082f52ad9dc355291/lib/cext/include/ruby/internal/intern/struct.h#L47-L53 so Rust can't find them. For such variadic functions we need to declare them like that otherwise it doesn't work on the upcall (NFI->Sulong) and there is no way to pass the varargs as-is. I can't think of an easy solution for that, except rb-sys not exposing these functions on TruffleRuby. Although maybe the generated code in trampoline.c (by tool/generate-cext-trampoline.rb) could define the variadic functions and automatically forward to the va_list variants? That's complicated but it seems possible and worth a try (IIRC there was some subtle issue with NFI and .../va_list I'll need to check). EDIT: we could also manually handle those in tool/generate-cext-trampoline.rb for simplicity.

And there are still some r_basic.as_ref().flags usages in rb-sys which should be changed/skipped to support truffleruby.

eregon avatar Jan 09 '25 16:01 eregon

It looks like there are two open PRs for rb-sys for better TruffleRuby support (which probably need to be addressed before Magnus can support it).

  • https://github.com/oxidize-rb/rb-sys/pull/467
  • https://github.com/oxidize-rb/rb-sys/pull/516

ankane avatar May 05 '25 22:05 ankane

I forgot to write here but TruffleRuby is already in rb-sys/rb_sys CI: https://github.com/oxidize-rb/rb-sys/blob/8548a8b6369494a4963eeeb6e08b918b0919fe08/.github/workflows/ci.yml#L48-L55

So at least some gems depending on rb-sys like blake3-rb already work fine. In general I don't think these 2 PRs are needed for gems using rb_sys for TruffleRuby, I think they are only needed if the gem specifically uses these functions from Rust.

eregon avatar May 06 '25 12:05 eregon

From the issues linked in the PRs, they're both needed for Magnus, which the gems mentioned in this thread use (commonmarker and polars-df).

  • https://github.com/oxidize-rb/rb-sys/issues/448
  • https://github.com/oxidize-rb/rb-sys/issues/447

ankane avatar May 06 '25 19:05 ankane

Right probably all of https://github.com/oxidize-rb/rb-sys/milestone/2 is needed for Magnus.

eregon avatar May 07 '25 10:05 eregon