Gem installation fails on MacOS

[collimarco]

trafficstars

I cannot install this gem on MacOS (Monterey / Intel / Ruby 3.1.2).

This is the error message:

$ brew install rocksdb
Warning: rocksdb 7.5.3 is already installed and up-to-date.
To reinstall 7.5.3, run:
  brew reinstall rocksdb
$ gem install rocksdb-ruby
Building native extensions. This could take a while...
ERROR:  Error installing rocksdb-ruby:
	ERROR: Failed to build gem native extension.

    current directory: /Users/collimarco/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/rocksdb-ruby-1.0.2/ext/rocksdb
/Users/collimarco/.rbenv/versions/3.1.2/bin/ruby -I /Users/collimarco/.rbenv/versions/3.1.2/lib/ruby/3.1.0 -r ./siteconf20220907-11247-eq0d9k.rb extconf.rb
checking for -lrocksdb_debug... no
checking for rocksdb/db.h... no
can't find header or library of rocksdb
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
	--with-opt-dir
	--without-opt-dir
	--with-opt-include
	--without-opt-include=${opt-dir}/include
	--with-opt-lib
	--without-opt-lib=${opt-dir}/lib
	--with-make-prog
	--without-make-prog
	--srcdir=.
	--curdir
	--ruby=/Users/collimarco/.rbenv/versions/3.1.2/bin/$(RUBY_BASE_NAME)
	--with-rocksdb-dir
	--without-rocksdb-dir
	--with-rocksdb-include
	--without-rocksdb-include=${rocksdb-dir}/include
	--with-rocksdb-lib
	--without-rocksdb-lib=${rocksdb-dir}/lib
	--with-rocksdb_debug-dir
	--without-rocksdb_debug-dir
	--with-rocksdb_debug-include
	--without-rocksdb_debug-include=${rocksdb_debug-dir}/include
	--with-rocksdb_debug-lib
	--without-rocksdb_debug-lib=${rocksdb_debug-dir}/lib
	--with-rocksdb_debuglib
	--without-rocksdb_debuglib

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  /Users/collimarco/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/extensions/x86_64-darwin-21/3.1.0/rocksdb-ruby-1.0.2/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in /Users/collimarco/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/rocksdb-ruby-1.0.2 for inspection.
Results logged to /Users/collimarco/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/extensions/x86_64-darwin-21/3.1.0/rocksdb-ruby-1.0.2/gem_make.out

I would love to use this gem and any help would be appreciated.

[collimarco]

I have also tried to run this code before the gem install as suggested here:

export CPATH="$CPATH:/usr/local/Cellar/rocksdb/7.5.3/include"
export LIBRARY_PATH="$LIBRARY_PATH:/usr/local/Cellar/rocksdb/7.5.3"

However it doesn't work and I get the exact same error... Any idea?

[collimarco]

I tried to downgrade from the latest version of Ruby to version 2.7 and the first check about the headers and libraries passes, however the build still fails with a different build error:

/usr/local/Cellar/rocksdb/7.5.3/include/rocksdb/wide_columns.h:49:21: error: no template named 'make_from_tuple' in namespace 'std'; did you mean 'make_tuple'?

So I guess that this is not a problem specific to my computer, but it's the gem that is not compatible with the current versions of Ruby and rocksdb :(

[katafrakt]

FWIW I can reproduce both errors on M1 Mac and on Linux, so it's a more general problem.

[katafrakt]

There are two things happening here.

First is that indeed gem installation process does not see a directory with rocksdb header files. This can be fixed by passing it like this:

gem install rocksdb-ruby -- --with-rocksdb-dir="/opt/homebrew/Cellar/rocksdb/7.5.3"

However, there is a second problem here, which manifests in the log with this:

In file included from conftest.c:3:
/opt/homebrew/Cellar/rocksdb/7.5.3/include/rocksdb/db.h:14:10: fatal error: 'map' file not found
#include <map>
         ^~~~~
1 error generated.
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: #include <rocksdb/db.h>
/* end */

As far as I understand, this is because mkmf uses clang to check for the library existence, but should use clang++ instead. Unfortunately, I don't know how to force it to use clang++ instead.

[collimarco]

@katafrakt Have you finally found any solution? Or are you also stuck on this?

[katafrakt]

If you run

gem install rocksdb-ruby -- --with-rocksdb-dir="/opt/homebrew/Cellar/rocksdb/7.5.3"

with Ruby 3.0.2, it works correctly, at least for me (the actual path might be different on Intel Mac).

So, I'm pretty sure this is Ruby's fault. They changed something in mkmf and I don't see it documented anywhere (classic Ruby development).

[collimarco]

@katafrakt Unfortunately it doesn't work for me on MacOS Intel (even if I adapt the Homebrew path and even using Ruby 3.0.x):

$ rbenv shell 3.0.4
$ gem install rocksdb-ruby -- --with-rocksdb-dir="/usr/local/Cellar/rocksdb/7.5.3"
Fetching rocksdb-ruby-1.0.2.gem
Building native extensions with: '--with-rocksdb-dir=/usr/local/Cellar/rocksdb/7.5.3'
This could take a while...
ERROR:  Error installing rocksdb-ruby:
	ERROR: Failed to build gem native extension.

    current directory: /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/rocksdb-ruby-1.0.2/ext/rocksdb
/Users/collimarco/.rbenv/versions/3.0.4/bin/ruby -I /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/3.0.0 -r ./siteconf20220910-2104-2e6rwr.rb extconf.rb --with-rocksdb-dir\=/usr/local/Cellar/rocksdb/7.5.3
checking for -lrocksdb_debug... no
checking for rocksdb/db.h... yes
checking for -lrocksdb... yes
creating Makefile

current directory: /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/rocksdb-ruby-1.0.2/ext/rocksdb
make DESTDIR\= clean

current directory: /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/rocksdb-ruby-1.0.2/ext/rocksdb
make DESTDIR\=
compiling rocksdb_batch_rb.cc
In file included from rocksdb_batch_rb.cc:1:
In file included from ./rocksdb_batch_rb.h:1:
In file included from /usr/local/Cellar/rocksdb/7.5.3/include/rocksdb/db.h:27:
In file included from /usr/local/Cellar/rocksdb/7.5.3/include/rocksdb/transaction_log.h:12:
In file included from /usr/local/Cellar/rocksdb/7.5.3/include/rocksdb/write_batch.h:36:
In file included from /usr/local/Cellar/rocksdb/7.5.3/include/rocksdb/write_batch_base.h:14:
/usr/local/Cellar/rocksdb/7.5.3/include/rocksdb/wide_columns.h:48:20: error: no template named 'make_from_tuple' in namespace 'std'; did you mean 'make_tuple'?
      : name_(std::make_from_tuple<Slice>(std::forward<NTuple>(name_tuple))),
              ~~~~~^~~~~~~~~~~~~~~
                   make_tuple
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/tuple:1259:1: note: 'make_tuple' declared here
make_tuple(_Tp&&... __t)
^
In file included from rocksdb_batch_rb.cc:1:
In file included from ./rocksdb_batch_rb.h:1:
In file included from /usr/local/Cellar/rocksdb/7.5.3/include/rocksdb/db.h:27:
In file included from /usr/local/Cellar/rocksdb/7.5.3/include/rocksdb/transaction_log.h:12:
In file included from /usr/local/Cellar/rocksdb/7.5.3/include/rocksdb/write_batch.h:36:
In file included from /usr/local/Cellar/rocksdb/7.5.3/include/rocksdb/write_batch_base.h:14:
/usr/local/Cellar/rocksdb/7.5.3/include/rocksdb/wide_columns.h:49:21: error: no template named 'make_from_tuple' in namespace 'std'; did you mean 'make_tuple'?
        value_(std::make_from_tuple<Slice>(std::forward<VTuple>(value_tuple))) {
               ~~~~~^~~~~~~~~~~~~~~
                    make_tuple
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/tuple:1259:1: note: 'make_tuple' declared here
make_tuple(_Tp&&... __t)
^
2 errors generated.
make: *** [rocksdb_batch_rb.o] Error 1

make failed, exit code 2

Gem files will remain installed in /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/rocksdb-ruby-1.0.2 for inspection.
Results logged to /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/extensions/x86_64-darwin-21/3.0.0/rocksdb-ruby-1.0.2/gem_make.out

Any idea?

[collimarco]

Ok, I made it, but it was a pain (and doesn't work on the latest ruby version):

1. Download the master branch from GitHub, the latest version on RubyGems is outdated and doesn't work
2. Use Ruby 3.0.4 because this gem doesn't work with Ruby 3.1 or newer (it always fails with the error "can't find header or library of rocksdb", even if you use --with-rocksdb-dir)
3. Run gem build rocksdb-ruby.gemspec in the folder downloaded from Github
4. Finally run a gem installation passing some additional params: gem install rocksdb-ruby-1.0.2.gem -- --with-rocksdb-dir="/usr/local/Cellar/rocksdb/7.5.3"

[collimarco]

The gem is installed successfully and I can see it:

$ gem list --local | grep rocksdb
rocksdb-ruby (1.0.2)

However for some obscure reason it still doesn't work (require 'rocksdb' always fails).

$ ruby rocks.rb 
<internal:/Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- rocksdb (LoadError)
	from <internal:/Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from rocks.rb:2:in `<main>'

[katafrakt]

Makes sense with installation. Changes in #30 fixed issues with make_from_tuple, but they have not yet been published to Rubygems. I haven't noticed that 🤦‍♂️

I cannot reproduce problems with require 'rocksdb' though.

[collimarco]

I can now install this gem successfully from RubyGems (thanks for releasing the new version!):

gem install rocksdb-ruby -- --with-rocksdb-dir="/usr/local/Cellar/rocksdb/7.5.3"

This is successful for Ruby 3.0, but you should still check compatibility with Ruby 3.1.

[katafrakt]

@collimarco I created PR #33 which should address it. Would appreciate if you could test it as well.

[irridescentrambler]

I can now install this gem successfully from RubyGems (thanks for releasing the new version!):

gem install rocksdb-ruby -- --with-rocksdb-dir="/usr/local/Cellar/rocksdb/7.5.3"

This is successful for Ruby 3.0, but you should still check compatibility with Ruby 3.1.

This worked for me. Just make sure, the directory location is right on your machine. On my machine the directory location is /opt/homebrew/Cellar/rocksdb/8.5.3

[jtippett]

Can confirm the above works for me on 3.2.2 with gem install rocksdb-ruby -- --with-rocksdb-dir="/opt/homebrew/Cellar/rocksdb/8.6.7"