xapian-ruby: compilation ends with fatal error: 'graalvm/llvm/polyglot.h' file not found

[deepj]

truffleruby 22.2.0-dev-0f63df6f, like ruby 3.0.3, GraalVM CE Native [aarch64-darwin]

To reproduce:

gem install xapian-ruby

Error:

Fetching xapian-ruby-1.4.18.gem
Building native extensions. This could take a while...
ERROR:  Error installing xapian-ruby:
	ERROR: Failed to build gem native extension.

    current directory: ~/.gem/truffleruby/3.0.3/gems/xapian-ruby-1.4.18
~/.rubies/truffleruby-dev/bin/ruby -I~/.rubies/truffleruby-dev/lib/ruby/site_ruby/3.0.3.6 -rrubygems ~/.gem/truffleruby/3.0.3/gems/rake-13.0.6/exe/rake RUBYARCHDIR\=~/.gem/truffleruby/3.0.3/extensions/aarch64-darwin/3.0.3.6/xapian-ruby-1.4.18 RUBYLIBDIR\=~/.gem/truffleruby/3.0.3/extensions/aarch64-darwin/3.0.3.6/xapian-ruby-1.4.18
tar -xJf xapian_source/xapian-core-1.4.18.tar.xz
tar -xJf xapian_source/xapian-bindings-1.4.18.tar.xz

...

/Library/Developer/CommandLineTools/usr/bin/make  all-am
depbase=`echo xapian_wrap.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
	/bin/sh ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..  -I~/.rubies/truffleruby-dev/lib/cext/include -I~/.rubies/truffleruby-dev/lib/cext/include  -fno-strict-aliasing -Wall -Wno-unused -Wno-uninitialized -fvisibility=hidden -I~/.gem/truffleruby/3.0.3/gems/xapian-ruby-1.4.18/xapian-core-1.4.18/include -g -O2 -std=gnu++11 -MT xapian_wrap.lo -MD -MP -MF $depbase.Tpo -c -o xapian_wrap.lo xapian_wrap.cc &&\
	mv -f $depbase.Tpo $depbase.Plo
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -I~/.rubies/truffleruby-dev/lib/cext/include -I~/.rubies/truffleruby-dev/lib/cext/include -fno-strict-aliasing -Wall -Wno-unused -Wno-uninitialized -fvisibility=hidden -I~/.gem/truffleruby/3.0.3/gems/xapian-ruby-1.4.18/xapian-core-1.4.18/include -g -O2 -std=gnu++11 -MT xapian_wrap.lo -MD -MP -MF .deps/xapian_wrap.Tpo -c xapian_wrap.cc  -fno-common -DPIC -o .libs/xapian_wrap.o
In file included from xapian_wrap.cc:24:
In file included from ~/.rubies/truffleruby-dev/lib/cext/include/ruby.h:38:
In file included from ~/.rubies/truffleruby-dev/lib/cext/include/ruby/ruby.h:15:
In file included from ~/.rubies/truffleruby-dev/lib/cext/include/ruby/internal/config.h:147:
~/.rubies/truffleruby-dev/lib/cext/include/truffleruby/truffleruby-pre.h:25:10: fatal error: 'graalvm/llvm/polyglot.h' file not found
#include <graalvm/llvm/polyglot.h>
         ^~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make[3]: *** [xapian_wrap.lo] Error 1
make[2]: *** [all] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
rake aborted!

~/.gem/truffleruby/3.0.3/gems/xapian-ruby-1.4.18/rakefile:9:in `system!'
~/.gem/truffleruby/3.0.3/gems/xapian-ruby-1.4.18/rakefile:40:in `block (2 levels) in <top (required)>'
<internal:core> core/dir.rb:291:in `chdir'
~/.gem/truffleruby/3.0.3/gems/xapian-ruby-1.4.18/rakefile:35:in `block in <top (required)>'
~/.gem/truffleruby/3.0.3/gems/rake-13.0.6/exe/rake:27:in `<main>'
Tasks: TOP => default
(See full trace by running task with --trace)

rake failed, exit code 1

Gem files will remain installed in ~/.gem/truffleruby/3.0.3/gems/xapian-ruby-1.4.18 for inspection.
Results logged to ~/.gem/truffleruby/3.0.3/extensions/aarch64-darwin/3.0.3.6/xapian-ruby-1.4.18/gem_make.out

Full log output: https://gist.github.com/deepj/7f6ea3e6113d1d857b3ab339c8e5fa0d

[eregon]

It looks like compilation fails because it does not use the GraalVM LLVM toolchain. The reason seems to be it's not a usual extension (see files at https://github.com/garaio/xapian-ruby) and instead of an extconf.rb it's a Rakefile: https://github.com/garaio/xapian-ruby/blob/master/Rakefile That doesn't require mkmf, and that's what sets up the GraalVM LLVM toolchain in PATH: https://github.com/oracle/truffleruby/blob/07df635e4734f17b29f55d75c1541154a1b4db36/lib/mri/mkmf.rb#L52

@deepj Could you try locally to add a require 'mkmf' in that Rakefile and see if that solves it?

[eregon]

Actually there is probably an even easier way to check if that solves it, with RUBYOPT=-rmkmf gem install xapian-ruby. I tried that, but it fails:

...
checking whether the g++ linker (/home/eregon/.rubies/truffleruby-dev/lib/sulong/native/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether g++ is a working C++ compiler... no
configure: error: 
*** You need a working C++ compiler to compile Xapian, but configure failed to
*** find one.  If you have a working C++ compiler, you can tell configure where
*** to find it by invoking it like so:
***
***   ./configure CXX=/opt/bin/c++

rake aborted!

Looking at xapian-core-1.4.18/config.log:

configure:16983: checking whether g++ is a working C++ compiler
configure:17013: g++ -o conftest -g -O2  -L/home/eregon/.rubies/truffleruby-dev/lib/gems/gems/xapian-ruby-1.4.18/lib conftest.cpp  >&5
configure:17013: $? = 0
configure:17013: ./conftest
./conftest: error while loading shared libraries: libc++.so.1: cannot open shared object file: No such file or directory
configure:17013: $? = 127
configure: program exited with status 127
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "xapian-core"
| #define PACKAGE_TARNAME "xapian-core"
| #define PACKAGE_VERSION "1.4.18"
| #define PACKAGE_STRING "xapian-core 1.4.18"
| #define PACKAGE_BUGREPORT "https://xapian.org/bugs"
| #define PACKAGE_URL ""
| #define PACKAGE "xapian-core"
| #define VERSION "1.4.18"
| #define STDC_HEADERS 1
| #define HAVE_SYS_TYPES_H 1
| #define HAVE_SYS_STAT_H 1
| #define HAVE_STDLIB_H 1
| #define HAVE_STRING_H 1
| #define HAVE_MEMORY_H 1
| #define HAVE_STRINGS_H 1
| #define HAVE_INTTYPES_H 1
| #define HAVE_STDINT_H 1
| #define HAVE_UNISTD_H 1
| #define HAVE_DLFCN_H 1
| #define LT_OBJDIR ".libs/"
| /* end confdefs.h.  */
| int main() {}
configure:17025: result: no
configure:17037: error:
*** You need a working C++ compiler to compile Xapian, but configure failed to
*** find one.  If you have a working C++ compiler, you can tell configure where
*** to find it by invoking it like so:
***
***   ./configure CXX=/opt/bin/c++

That's because it doesn't use mkmf. There is handling for this stuff in https://github.com/oracle/truffleruby/blob/07df635e4734f17b29f55d75c1541154a1b4db36/lib/mri/mkmf.rb#L2302-L2310