warbler Problem with bundler git gems

When I execute java -jar <JAR NAME> I get "LoadError: no such file to load" when I require any gems that are bundled from git.

When I run the same ruby code via bundle exec it works just fine.

Is this a bug or something Im doing wrong?

I'm using warbler (1.4.0.dev), bundler (1.3.5) and jruby-1.7.0

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/1446617-problem-with-bundler-git-gems?utm_campaign=plugin&utm_content=tracker%2F136961&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F136961&utm_medium=issues&utm_source=github).

Jun 10 '13 20:06 dallisonhcom

When you say "bundled from git" do you mean the :git => directive on a gem in your Gemfile? Do you have a sample bare-bones app that you can use to reproduce this? Thanks.

Jun 10 '13 23:06 jkutner

Thanks for the quick response. I've created a sample bare bones gem (https://github.com/davidpallisonexp/hola) and a simple bare bones app (https://github.com/davidpallisonexp/warbler-example) which I can reproduce the problem on. The steps I worked through are as follows :

rvm gemset create warbler // Create a new rvm gemset for testing rvm gemset use warbler // Switch to clean gemset gem install bundler // Install bundler gem install warbler // Install Gem built from master bundle install // Install Gems for example app bundle exec ruby bin/hello.rb // Run example app via bundle exec which works just fine Hello world!

warble // Create jar file using warbler java -jar warbler-example.jar // Run app from jar file LoadError: no such file to load -- hola require at org/jruby/RubyKernel.java:1054 require at jar:file:/var/folders/18/sqzlph9n0vv6phtr3136twlh536zyf/T/jruby6942042007361873674extract/jruby-stdlib-1.7.4.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/custom_require.rb:36 (root) at file:/Users/dallison/Hcom/Development-DevInf/warbler.bug2/warbler-example/warbler-example.jar!/warbler-example/bin/hello.rb:1 load at org/jruby/RubyKernel.java:1073 (root) at file:/Users/dallison/Hcom/Development-DevInf/warbler.bug2/warbler-example/warbler-example.jar!/META-INF/main.rb:1 require at org/jruby/RubyKernel.java:1054 (root) at file:/Users/dallison/Hcom/Development-DevInf/warbler.bug2/warbler-example/warbler-example.jar!/META-INF/main.rb:1 (root) at jar:file:/var/folders/18/sqzlph9n0vv6phtr3136twlh536zyf/T/jruby6942042007361873674extract/jruby-stdlib-1.7.4.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/custom_require.rb:1 error: org.jruby.embed.EvalFailedException: (LoadError) no such file to load -- hola

I can see the hola gem under bundler/gems in the archive so it looks like it is packaged but in the wrong place or not on the load path? Or more likely, am I doing something wrong?

Thanks for your help with this.

Jun 11 '13 19:06 dallisonhcom

I see similar issues with a WAR. The :git gem is stored in WEB-INF/gems/bundler/gems/, whereas all the "normal" gems are in WEB-INF/gems/gems/. Loading the app results in the following error:

org.jruby.rack.RackInitializationException: git://github.com/shepmaster/select2-rails.git (at ie8-hack) is not checked out. Please run `bundle install`

Jul 23 '13 16:07 shepmaster

For my case, I was running my Java app server with GEM_HOME set to my local development environment. The thing that tipped me off was that my stacktrace had references to /home/user/.rvm/, which didn't make any sense at first.

Jul 23 '13 18:07 shepmaster

I get this same problem. It seems it seems to work when I change the following in the META-INF/init.rb:

ENV['GEM_HOME'] = File.expand_path('../../WEB-INF', __FILE__)

to

ENV['GEM_HOME'] ||= File.expand_path('../../WEB-INF', __FILE__)

or setting config.override_gem_home = false

I also need to change: ENV['BUNDLE_GEMFILE'] = 'blah' to ENV['BUNDLE_GEMFILE'] ||= 'blah'

Oct 01 '13 15:10 nerdrew

@nerdrew what version of warbler are you using?

Oct 01 '13 16:10 jkutner

I'm using 1.4.0.beta1.

Oct 01 '13 16:10 nerdrew

I'm not sure if this kind of spec belongs in the integration tests. If so, I can setup maven and look at it.

Oct 01 '13 17:10 nerdrew

This is fine in the specs.

Oct 01 '13 17:10 jkutner

This is blocking us from upgrading to warbler 1.4.0. Was this fix good? Are there any plans to merge it?

Jan 11 '14 02:01 vegetabill

It looks like there was just never a pull request issued for the change. I'll be happy to get that in and cut a release if you can wrangle one up.

Jan 11 '14 02:01 jkutner

Ooops. I think I force pushed over my branch. I opened a new PR for this: https://github.com/jruby/warbler/pull/231

Jan 11 '14 22:01 nerdrew

These fixes made git bundled gems working for me for the first time as well.

Is there an ETA for a release of 1.4.1?

Feb 12 '14 22:02 LFDM

Yea, I can get one out tonight. I was hoping to get some other things fixed and add to the 1.4.1 release, but haven't made any progress on those. I'll backlog them.

Feb 12 '14 22:02 jkutner

I wouldn't be opposed to that, but take all the time you need! :) Thanks in any event.

Feb 12 '14 22:02 LFDM

I'm closing this. If it pops up again in 1.4.1 we can just reopen this issue.

Feb 14 '14 15:02 jkutner

Has anyone verified this worked for them? I am having what appears to be the same issue on 1.4.1.

Feb 19 '14 21:02 bbradbury

Ugh, this seems to be a very delicate matter. It works for me...

Feb 19 '14 21:02 LFDM

So, working to diagnose; definitely is a problem only with :git gems. ENV['GEM_HOME'] is set to the jar root when printed from inside the program before the first require. The git gems end up at a different spot in the jar than the other gems (e.g. bundler/gems/jcouchbase-4fa63b91f89f/ vs gems/mongo-1.9.2/), and the git jars' gemspec files do not end up in the jar (e.g. specifications/mongo-1.9.2.gemspec vs nothing for the jcouchbase gem). Could that be related?

Another possible aggravating factor is that I am using both a gemspec (to define an executable in bin/) and a Gemfile, with most dependencies declared only in the Gemfile-- which warbler seems to be fine with in general. It was ambiguous if that was ok or not though-- is that totally the wrong thing to do?

Feb 19 '14 22:02 bbradbury

Could it be the fix above fixed the war case but not the jar case?

Feb 19 '14 22:02 bbradbury

Yes, that appears to be likely: https://github.com/jruby/warbler/pull/231/files https://github.com/jruby/warbler/blob/master/lib/warbler/templates/jar.erb

Can you test locally and confirm? If so, I can get that fixed this evening, and we're likely to have another release soon.

Feb 19 '14 22:02 jkutner

Yes, testing now. I'm going to patch a local warbler fork, so I might end up with a pull request.

Feb 19 '14 22:02 bbradbury

that would be awesomesauce. thx

Feb 19 '14 22:02 jkutner

So, adding the ||= didn't make a difference, so I'm dead in the water for the moment. The value of ENV['BUNDLE_GEMFILE'] was already set to the full path of the Gemfile on my file system. It seems to me that the gem search path just can't find the gems in /bundler/gems/ -- it's as if it's only searching /gems (and the git gems aren't there-- are they supposed to be?). config.override_gem_home didn't make any difference, true or false.

Feb 19 '14 23:02 bbradbury

I can feel your pain. I tried to fix some of that pre-1.4.1 (only for war files though) and basically ended up doing it trial and error style. Haven't spend much time investigating this, but never really understood how these gem search paths inside of warbler really work :(

Feb 19 '14 23:02 LFDM

So, I think what the problem is is that the bundler gems end up inside '/bundler/gems'. The templates' init.rb output only sets ENV['GEM_HOME']-- not to be confused with ENV['GEM_PATH'], which I don't think warbler currently deals with. GEM_HOME is a singular path; GEM_PATH is a comma delimited list of gem locations.

Since GEM_HOME points to /gems, either the bundler git gems need to end up there, or init.rb needs to set ENV['GEM_PATH'] to include both the current value of GEM_HOME and the bundler gems path (/bunder/gems).

Refs:

Feb 20 '14 00:02 bbradbury

I ended up looking at the exact same piece of code back in the day. I'm afraid was too lazy to ask... @jkutner, maybe you can explain: What's the rationale of putting the bundled gems in a separate folder in the first place? They are so prominently placed there, I imagine there must be a reason for that. If not - well, then it might really be as easy as placing all gems inside the regular gems folder.

Feb 20 '14 00:02 LFDM

Had success finding the git gems by patching bundler.rb#L60-63 to

          case spec.source
          when ::Bundler::Source::Git
            config.gems << spec
          #  config.bundler[:git_specs] ||= []
          #  config.bundler[:git_specs] << spec

...and thus treating the git bundled gems as normal ones. I noticed that I lost the git sha in the version, and they reverted to their original reported version (so jcouchbase-1.1.3 vs jcouchbase-012345abcd), which may not be desirable. Things still break down for me around jars packaged in gems-- though that might be entirely outside the scope of this bug, assuming the special handling has nothing to do with dependent jars packaged with gems.

Feb 20 '14 01:02 bbradbury

Did some more reading-- I think that what warbler is currently trying to accomplish is layout git gems internally the same way Bundler does, so that Bunder.setup and Bundler.require work right. I switched back to an unadulterated 1.4.1, and added

require 'bundler'
Bundler.setup(:default)

at the top of the file in the bin/ directory that launches my app. I think this is what's supposed to work (because I believe Rails effectively ends up doing this in the war case), but it generates a LoadError: no such file to load -- bundler/index.

There's a discussion along these lines here: https://www.ruby-forum.com/topic/4416022

Feb 20 '14 18:02 bbradbury

I think https://github.com/jruby/warbler/issues/235 is the same issue.

Mar 06 '14 20:03 jwinter