chruby icon indicating copy to clipboard operation
chruby copied to clipboard

Use ~/.gem/$ruby/X.Y as the GEM_HOME

Open postmodern opened this issue 11 years ago • 36 comments

Now that Ruby has moved to Semantic Versioning it means each new release will have a unique X.Y.Z version. Since GEM_HOME is defined as ~/.gem/$ruby/X.Y.Z, this means each new release will have an empty gem dir, forcing users to re-install all gems.

Would it be safe to define GEM_HOME as ~/.gem/$ruby/X.Y, ensuring that gems are shared between patch releases?

postmodern avatar Dec 28 '13 02:12 postmodern

There could be problems with gems that used internal C functions that were added in specific 1.9.x versions.

postmodern avatar Dec 28 '13 02:12 postmodern

However, RubyGems 2.2.0 how keeps built C extensions in a separate extensions/ directory:

extensions/
`-- x86_64-linux
    `-- 2.1.0-static
         `-- bond-0.4.3
            |-- gem.build_complete
            `-- gem_make.out

postmodern avatar Dec 28 '13 02:12 postmodern

The change might be as simple as export GEM_HOME="$HOME/.gem/$RUBY_ENGINE/${RUBY_VERSION%.*}".

postmodern avatar Dec 28 '13 03:12 postmodern

Another blocker is that rubygems has generated fully qualified #! for it's binstubs, instead of the usual #!/usr/bin/env ruby. This makes it difficult to share rubygems between multiple Rubies.

postmodern avatar Jan 03 '14 02:01 postmodern

Unfortunately there is no GEMOPT env variable we could set to enable the --env-shebang option by default.

postmodern avatar Jan 03 '14 02:01 postmodern

We just ran into the fully qualified shebang issue ourselves when a coworker updated his version of ruby-2.0.0 and deleted the older version. Maybe it makes sense to just suggest in the README to include "--env-shebang" in the install/update parameters of gemrc?

As for removing one identifier level from ~/.gem/${ruby}/X.Y.Z, it seems slightly premature, given that 1.9 has not yet reached EOL and is distributed with Rubygems 1.8.

ilikepi avatar Feb 24 '14 23:02 ilikepi

Probably a good idea to document --env-shebang and gem pristin. Long-term, I wish #!/usr/bin/env ruby would be the default, and #!/path/to/ruby would only be used when ruby has a custom suffix (ex: ruby1.8).

postmodern avatar Feb 25 '14 01:02 postmodern

I think it would make sense to use the ruby C API version for the directories, which would be 1.8 for ruby 1.8.x, 1.9.1 for ruby 1.9.1+, 2.0.0 for 2.0.x and 2.1.0 for 2.1.x. All gems compiled against the same API version are supposed to be binary compatible.

The c api version can be seen by looking at usr/include/ruby/version.h and the macros RUBY_API_VERSION_MAJOR, RUBY_API_VERSION_MINOR and RUBY_API_VERSION_TEENY, but I don't know how to get it with ruby code. This also doesn't really work for implementations without C api, like jruby.

felixbuenemann avatar May 14 '14 22:05 felixbuenemann

This is probably interesting to the discussion: Introducing a semantic versioning scheme and branching policy

felixbuenemann avatar May 14 '14 22:05 felixbuenemann

The following code returns the C API version:

require 'rbconfig'
puts RbConfig::CONFIG["ruby_version"]

On jruby this returns "1.9" while RUBY_VERSION returns "2.0.0", which might be because C-Extension support was removed in newer versions.

felixbuenemann avatar May 14 '14 22:05 felixbuenemann

Unfortunately using the API version would mean no gem separation between different versions. Also keep in mind that rubygems now generates an absolute #!/path/to/ruby for gem executables.

postmodern avatar May 14 '14 22:05 postmodern

Isn't no gem separation between minor versions the point of the discussion? If upgrading rubies gem pristine --all --only-executables should do the trick. If sharing between rubyies with same API version is desired, then I guess --env-shebang in gemrc would do the trick.

felixbuenemann avatar May 14 '14 22:05 felixbuenemann

Ah it appears ruby-core is bumping the API version on each minor release, instead of having one API version for all 2.x.y releases. This might work after all.

I still wish there was an ENV variable we could set to enable --env-shebang by default.

postmodern avatar May 14 '14 22:05 postmodern

Yes, that would be nice. For now I'm using this patch with 0.3.8:

--- chruby/0.3.8/share/chruby/chruby.sh 2014-05-15 00:48:18.000000000 +0200
+++ chruby/0.3.8/share/chruby/chruby.sh 2014-05-15 00:39:08.000000000 +0200
@@ -44,14 +44,16 @@

    eval "$("$RUBY_ROOT/bin/ruby" - <<EOF
 begin; require 'rubygems'; rescue LoadError; end
+begin; require 'rbconfig'; rescue LoadError; end
 puts "export RUBY_ENGINE=#{defined?(RUBY_ENGINE) ? RUBY_ENGINE : 'ruby'};"
 puts "export RUBY_VERSION=#{RUBY_VERSION};"
+puts "export RUBY_API_VERSION=#{RbConfig::CONFIG["ruby_version"] rescue RUBY_VERSION}"
 puts "export GEM_ROOT=#{Gem.default_dir.inspect};" if defined?(Gem)
 EOF
 )"

    if (( $UID != 0 )); then
-       export GEM_HOME="$HOME/.gem/$RUBY_ENGINE/$RUBY_VERSION"
+       export GEM_HOME="$HOME/.gem/$RUBY_ENGINE/$RUBY_API_VERSION"
        export GEM_PATH="$GEM_HOME${GEM_ROOT:+:$GEM_ROOT}${GEM_PATH:+:$GEM_PATH}"
        export PATH="$GEM_HOME/bin${GEM_ROOT:+:$GEM_ROOT/bin}:$PATH"
    fi

felixbuenemann avatar May 14 '14 22:05 felixbuenemann

Actually you could use Gem.ruby_api_version.

postmodern avatar May 14 '14 22:05 postmodern

Actually Gem.ruby_api_version was added sometime in 2.x.

postmodern avatar May 14 '14 23:05 postmodern

One problem with using API version is lack of separation between 1.9.1, 1.9.2, 1.9.3.

postmodern avatar May 14 '14 23:05 postmodern

We could wait until 1.9.x is EoLed and then switch to using the API version.

postmodern avatar May 14 '14 23:05 postmodern

Ah, that's nicer.

I grepped through the rubygems source, and there seems to be an environment variable GEMRC that can point to additional gem config files, so it would be possible to inject an environment specific gemrc.

Apparently you can do something like: GEMRC=/path/to/nother/gemrc:$TMP/extra/gemrc

It should be possible to share gems between 1.9.1, 1.9.2 and 1.9.3 as they all have the same C api version.

felixbuenemann avatar May 14 '14 23:05 felixbuenemann

If you upgrade a debian system that's running eg. ruby-1.9.2-p0 as the system ruby to a newer release that runs eg. ruby-1.9.3-p194 you can continue using all your gems, cause the gems are binary compatible. So I don't understand why the GEMs should be separated if they are compatible.

felixbuenemann avatar May 14 '14 23:05 felixbuenemann

@felixbuenemann gems should be seperated by "version". Allowing gems from 1.9.x versions to mix would add an edge-case to how chruby separates gems. Also, rubygems 1.8.23 will generate absolute #! even if --env-shebang is included in ~/.gemrc.

postmodern avatar May 14 '14 23:05 postmodern

Hmm, maybe it would make sense to just detect the rubygems version and fallback to the full version scheme if it's too old?

felixbuenemann avatar May 14 '14 23:05 felixbuenemann

If we could somehow query the ruby version (sans the patch-level number) for both 1.9.x and 2.x, that would also work.

  • Given 1.9.3, gemdir version should be 1.9.3.
  • Given 2.1.0, gemdir version should be 2.1.

postmodern avatar May 14 '14 23:05 postmodern

I'm probably misunderstanding you, but that's already returned by RUBY_VERSION, the patchlevel is stored in RUBY_PATCHLEVEL.

felixbuenemann avatar May 14 '14 23:05 felixbuenemann

Er, by patch-level I mean RUBY_PATCHLEVEL wrt 1.9.x and the last number in RUBY_VERSION in 2.x.

postmodern avatar May 14 '14 23:05 postmodern

Ah, ok, so you mean the major releases like 1.9.2, 1.9.3, 2.0, 2.1, 2.2.

felixbuenemann avatar May 14 '14 23:05 felixbuenemann

Would this be sufficient?

RUBY_ENGINE == "ruby" && RUBY_VERSION.sub(/([2-9]\.[0-9])\.[0-9]/, '\1')

This could also be done in the shell (no bash expert here ;-):

$([[ $RUBY_ENGINE = ruby  &&  ${RUBY_VERSION%%.*} -ge 2 ]] && echo ${RUBY_VERSION%.*} || echo $RUBY_VERSION)

felixbuenemann avatar May 14 '14 23:05 felixbuenemann

+1 to use the ABI version, which also would play nicely with rubygems --user-install install option.

jhass avatar Sep 03 '14 21:09 jhass

What's the problem with using RbConfig::CONFIG["ruby_version"]? Ruby 1.9.2 maintenance has ended this year. Ruby 1.9.3 versions can share gems between patch-level releases. Ruby 2.0.x and 2.1.x will use 2.0.0 and 2.1.0 directories respectively.

semaperepelitsa avatar Oct 24 '14 00:10 semaperepelitsa

Until rubygems defaults to using #!/usr/bin/env ruby instead of an explicit #!/path/to/ruby, it's looking like separate gem directories for each ruby is the only way.

postmodern avatar Oct 24 '14 00:10 postmodern

Maybe it could be a configuration option for chruby? I have --env-shebang in my gemrc, so it's a small issue, but I understand that it makes it hard to make it the default.

felixbuenemann avatar Oct 24 '14 16:10 felixbuenemann

It's too bad this bug is still open without much action. It's very confusing, for example, trying to use Bundler with chruby, as bundle install puts my gems under .gem/ruby/2.2.0, when chruby expects to find them in .gem/ruby/2.2.2. It seems to make bundler and chruby practically incompatible.

wryfi avatar Sep 21 '15 17:09 wryfi

@wryfi that sounds like a bug in bundler. Double check that the head -n1 $(which bundle) points to 2.2.2, and not 2.2.0. bundler should install missing gems into whatever $GEM_HOME is set to.

$ chruby 2.2.2
$ export GEM_HOME="$PWD/.gem"
$ unset GEM_PATH
$ bundle install
Fetching gem metadata from https://rubygems.org/..
Fetching version metadata from https://rubygems.org/.
Using rake 10.4.2
Installing json 1.8.3 with native extensions
Using bundler 1.10.6
Bundle complete! 2 Gemfile dependencies, 3 gems now installed.
Use `bundle show [gemname]` to see where a bundled gem is installed.
$ ls .gem/gems/
json-1.8.3

Also please comment on the rubygems bug, since it is blocking this change.

postmodern avatar Sep 21 '15 19:09 postmodern

The shebang points to the ruby-2.2.2 I installed via ruby-install. I have never had ruby-2.2.0 installed on this system.

wryfi avatar Sep 21 '15 20:09 wryfi

Oh, I see what I was doing wrong. I was using --path option with bundle, which always adds the ABI version to its install path. Realizing that I can do what I need by manipulating GEM_HOME during my build process.

wryfi avatar Sep 23 '15 18:09 wryfi

@wryfi see also gem_home for pushing/poping gem dirs.

postmodern avatar Sep 23 '15 18:09 postmodern