thoughtbot
Running appraisal specs with Bundler >= 2.4.0 broken
Hey there 👋 I'm trying to send a PR (https://github.com/thoughtbot/appraisal/pull/219) and I'd like to add specs. But I'm unable to run the current specs. What am I missing?
I did this:
git clone [email protected]:thoughtbot/appraisal.git
bundle
rake fails with various other warnings but also the same error (no surprise as the Rakefile does not seem to do something special).
$ rspec ./spec/acceptance/appraisals_file_bundler_dsl_compatibility_spec.rb:4 fails like this:
Run options: include {:locations=>{"./spec/acceptance/appraisals_file_bundler_dsl_compatibility_spec.rb"=>[4]}}
/Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/source/git/git_proxy.rb:354:in `allowed_with_path': The git source ../gems/omelette is not yet checked out. Please run `bundle install` before trying to start your application (Bundler::GitError)
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/source/git/git_proxy.rb:78:in `revision'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/source/git.rb:242:in `revision'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/source/git.rb:116:in `install_path'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/source/path.rb:134:in `expanded_path'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/source/path.rb:175:in `load_spec_files'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/source/git.rb:219:in `load_spec_files'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/source/path.rb:108:in `local_specs'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/source/git.rb:185:in `specs'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/source/rubygems_aggregate.rb:35:in `block (2 levels) in build_index'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/source/rubygems_aggregate.rb:33:in `each'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/source/rubygems_aggregate.rb:33:in `block in build_index'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/index.rb:9:in `build'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/source/rubygems_aggregate.rb:30:in `build_index'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/source/rubygems_aggregate.rb:12:in `initialize'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/definition.rb:935:in `new'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/definition.rb:935:in `source_requirements'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/definition.rb:543:in `resolution_packages'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/definition.rb:525:in `resolver'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/definition.rb:600:in `start_resolution'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/definition.rb:311:in `resolve'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/definition.rb:553:in `materialize'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/definition.rb:203:in `specs'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/definition.rb:270:in `specs_for'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/runtime.rb:18:in `setup'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler.rb:162:in `setup'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/setup.rb:26:in `block in <top (required)>'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/ui/shell.rb:159:in `with_level'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/ui/shell.rb:111:in `silence'
from /Users/basti/.gem/ruby/3.3.0/gems/bundler-2.5.4/lib/bundler/setup.rb:26:in `<top (required)>'
from <internal:/Users/basti/.rubies/ruby-3.3.0/lib/ruby/site_ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:127:in `require'
from <internal:/Users/basti/.rubies/ruby-3.3.0/lib/ruby/site_ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:127:in `require'
from /Users/basti/.rubies/ruby-3.3.0/lib/ruby/site_ruby/3.3.0/rubygems.rb:1375:in `<top (required)>'
from <internal:gem_prelude>:2:in `require'
from <internal:gem_prelude>:2:in `<internal:gem_prelude>'
F
Failures:
1) Appraisals file Bundler DSL compatibility supports all Bundler DSL in Appraisals file
Failure/Error:
raise RuntimeError, <<-error_message.strip_heredoc
Command #{command.inspect} exited with status #{exitstatus}. Output:
#{output.gsub(/^/, ' ')}
error_message
RuntimeError:
Command "bundle install --local" exited with status 1. Output:
# ./spec/support/acceptance_test_helpers.rb:163:in `block (2 levels) in run'
# ./spec/support/acceptance_test_helpers.rb:155:in `block in run'
# ./spec/support/acceptance_test_helpers.rb:150:in `chdir'
# ./spec/support/acceptance_test_helpers.rb:150:in `in_test_directory'
# ./spec/support/acceptance_test_helpers.rb:154:in `run'
# ./spec/acceptance/appraisals_file_bundler_dsl_compatibility_spec.rb:100:in `block (2 levels) in <top (required)>'
Finished in 7.35 seconds (files took 0.22456 seconds to load)
1 example, 1 failure
Failed examples:
rspec ./spec/acceptance/appraisals_file_bundler_dsl_compatibility_spec.rb:4 # Appraisals file Bundler DSL compatibility supports all Bundler DSL in Appraisals file
I think it's a bundler incompatibility that you're seeing. I haven't been able to pin down exactly what it is, but I was working on this last week.
I can run bundle exec rspec on Ruby 3.3.0 and with:
$ bundle exec bundler --version
Bundler version 2.3.7
…and the tests pass, but different versions cause problems.
In addition, last week I was playing around with a branch I'd worked on ages ago for testing out GitHub Actions and seen the same.
Could you try replicating with one of the combinations (see the GHA branch), and see if that helps you?
That was a good hint, thanks. I was a little lost and couldn't really find a good starting point. I did some very corse search and found 2.3.27 to be working. I poked at some 2.4.x and 2.5.x versions, but was seeing the same error. I can try some more smart bisecting in the next days, if that helps. My guess is that there broken something between 2.3.27 (laest 2.3.x release) and 2.4.0.
Looking at https://github.com/rubygems/rubygems/blob/master/bundler/CHANGELOG.md#240-december-24-2022, I kind of have the suspicion that https://github.com/rubygems/rubygems/pull/4475 might be the reason. Bundler::Source::Git has been changed quite a bit, where the error message is coming from.
🤔
Forgot to mention: I tested bundler versions with this: bundle _2.4.0_ update --bundler && bundle && rspec ./spec/acceptance/appraisals_file_bundler_dsl_compatibility_spec.rb. Since bundler also manages itself, debugging things like that became rather easy :)
Ah, hah, nice! It'd be great if you could spend the time tracking down what might have changed.
Whilst you're doing that, I'd be keep to hear if you have any ideas of what we could do to make these things a bit easier to debug, too.
This is what I have so far.
Minimal acceptance test:
require 'spec_helper'
describe 'Debugging Bundler Issues' do
it 'does something' do
build_git_gems %w(omelette)
build_gemfile <<-Gemfile
source 'https://rubygems.org'
gem "omelette", git: "../gems/omelette"
Gemfile
run 'bundle install --local --verbose'
end
end
Trigger the error via bundle _2.5.6_ exec rspec ./spec/acceptance/bundler_debug_spec.rb.
When you then go into tmp/stage and run bundle install --local --verbose you'll get the same error, but also see stderr from bundler:
Retrying `git clone --bare --no-hardlinks --quiet --no-tags --depth 1 --single-branch -- file://../gems/omelette /Users/basti/.gem/ruby/3.3.0/cache/bundler/git/omelette-0232b7510a79059d96d8bcbfb4a82ad436173b63` due to error (2/4): Bundler::Source::Git::GitCommandError Git error: command `git clone --bare --no-hardlinks --quiet --no-tags --depth 1 --single-branch -- file://../gems/omelette /Users/basti/.gem/ruby/3.3.0/cache/bundler/git/omelette-0232b7510a79059d96d8bcbfb4a82ad436173b63` in directory /Users/basti/.gem/ruby/3.3.0/cache/bundler/git/omelette-0232b7510a79059d96d8bcbfb4a82ad436173b63 has failed.
fatal: '/gems/omelette' does not appear to be a git repository
fatal: Could not read from remote repository.
[…]
BINGO!
Here is what happens:
- Bundler is doing
git clone --bare --no-hardlinks --quiet --no-tags --depth 1 --single-branch -- file://../gems/omelette […], to clone the git repo of the gem to the bundler cache using a shallow clone. -
BUT the
file://scheme does not permit relative paths (see RFC8089 "The path component represents the absolute path to the file in the file system." and RFC1738). - The underlying problem is hidden because
AcceptanceTestHelpers#runis unfortunately using Kernel#` which only captures standard out. In case of errors when invoking bundler, this is quite useless.
What does that mean?
tl;dr: I think Bundler does not support local git repositories to be defined via Gemfile (or it just broken). The docs are pretty clear though: For gems out of local git repositories you are supposed to do bundle config set local.GEM_NAME /path/to/local/git/repository.
I'll try to dig through the Bundler changelog and PRs to see if this was intentional or not. I suspect that this was an unintended side-effect introduced by https://github.com/rubygems/rubygems/pull/4475. As it was never documented/recommended to use gems from local git repositories in a different way.
IMO, btw: It should either work, or Bundler should raise an error, that local git repos are not supported, instead of trying to feed an invalid file:// URI to git.
Hi there! I've opened an issue on bundler repo to get some help/insight : https://github.com/rubygems/rubygems/issues/7925
@tisba local git repos are supported/tested but with full paths, not relative paths : https://github.com/rubygems/rubygems/blob/master/bundler/spec/install/gemfile/git_spec.rb#L463-L479
@nickcharlton @tisba this isssue is solved by https://github.com/thoughtbot/appraisal/pull/229
can you please take a look?