Byebug doesn't play nice with bootsnap

[hashwin]

Problem description

Byebug works fine the first time I add a debugger statement, but then when any other code is changed, the next time the breakpoint is hit, it seems to stop in some internal rails/other gem code. The issue is only alleviated by restarting the server, which can become time consuming when trying to debug an issue.

Expected behavior

The breakpoint should be retained even after code is changed.

Actual behavior

The breakpoint works fine the first time, but as soon as code is changed and the breakpoint is hit again, it seems to stop in internal rails code.

Steps to reproduce the problem

Simple example in HomeController:

class HomeController < ApplicationController
  before_action :redirect_signed_in_user, only: :welcome

  def welcome; end

  def redirect_signed_in_user
    debugger
    redirect_to root_url if user_signed_in?
  end
end

Works as expected

[20, 29] in /Users/ashwinhegde/code/apps/o3_crm/app/controllers/home_controller.rb
   20:
   21:   private
   22:
   23:   def redirect_signed_in_user
   24:     debugger
=> 25:     redirect_to root_url if user_signed_in?
   26:   end
   27:

Change some code, any code. Next time the breakpoint is hit - some internal devise code:

[117, 126] in /Users/ashwinhegde/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/devise-4.4.3/lib/devise/controllers/helpers.rb
   117:             opts[:scope] = :#{mapping}
   118:             warden.authenticate!(opts) if !devise_controller? || opts.delete(:force)
   119:           end
   120:
   121:           def #{mapping}_signed_in?
=> 122:             !!current_#{mapping}
   123:           end
   124:
   125:           def current_#{mapping}
   126:             @current_#{mapping} ||= warden.authenticate(scope: :#{mapping})
(byebug)

[deivid-rodriguez]

Thanks for the report, I've started seeing the same thing recently too... And I have no idea why. It seems like a regression to me. Can you try 10.0.0, 10.0.1, and 10.0.2 and compare?

[deivid-rodriguez]

@hashwin Are you using bootsnap by any chance? I think byebug does not play nice with bootsnap.

[hashwin]

@deivid-rodriguez Yes! Removing it seems to resolve the issue. Thank you so much for tracking this down!

[deivid-rodriguez]

@hashwin The following setup in config/boot.rb seems to work. It's the default config recommended in bootsnap's README, but changing compile_cache_iseq to false

require 'bootsnap'

env = ENV['RAILS_ENV'] || 'development'

Bootsnap.setup(
  cache_dir: 'tmp/cache',
  development_mode: env == 'development',
  load_path_cache: true,
  autoload_paths_cache: true,
  disable_trace: true,
  compile_cache_iseq: false,
  compile_cache_yaml: true
)

I definitely need to at least document this, so I'll reopen this issue and use it as a reminder.

[hashwin]

Thanks @deivid-rodriguez. I will try this and let you know if I run into more trouble.

[tzmfreedom]

Bootsnap use RubyVM::InstructionSequence.load_from_binary to load other file quickly, and byebug use TracePoint feature.

I guess TracePoint hook doesn't work when we load file with RubyVM::InstructionSequenc.load_from_binary on Ruby 2.5. So I posted the issue to Ruby Issue Tracking System. https://bugs.ruby-lang.org/issues/14702

I think this is why the example on https://github.com/deivid-rodriguez/byebug/issues/452#issuecomment-381358106 work correctly.

[francoisjacques]

I have the same issue on an older version of rails where bootsnap was integrated. It uses the same configuration as recommended above and it still breaks in a different spot than where the byebug statement is. If I strictly use pry (byebug and pry-byebug are removed from the gemfile) then it breaks at the right place. But I've lost the debugger, which is slightly annoying.

If there's anything I can do to help this issue being fixed (diagnosis, etc), let me know!

EDIT: from my Gemfile:

gem 'byebug', '= 10.0.2', :require => false
gem 'pry', '= 0.11.3', :require => false
gem 'pry-byebug', '= 3.6.0', :require => false

There's no other pry plugin.

[deivid-rodriguez]

Does it get fixed if you remove bootsnap?

[francoisjacques]

If I remove Bootsnap.setup, to be precise. The gem can be required, as long as Bootsnap.setup isn't called.

That's my workaround for now - skip Bootsnap.setup if a environment variable is defined. Not ideal, but it works. I'm sure other folks will hit that shortly.

[deivid-rodriguez]

I honestly have no idea why the custom bootsnap configuration is not working for you. I recommend that you put together some reproduction steps for your problems since the workaround seems to work for other people.

By the way, in case someone wants to help here, it might be worth exploring what debase is doing so that this works out of the box: https://github.com/denofevil/debase/pull/61.

[francoisjacques]

Oh, we're loading an old version of debase as well for interactive debugging from IDEs. I'll see if bumping its version helps.

[francoisjacques]

Actually, looking at the change in debase, it's perhaps due to the fact that ruby is used is still at 2.3? In mean, the custom bootsnap configuration might be working for 2.5 and later only?

[deivid-rodriguez]

Maybe... The issue @tzmfreedom posted to ruby-core is 2.5 specific, not sure if that could be related.

[francoisjacques]

I'll have to put this on the backburner for now, at least I've isolated that removing Bootsnap.setup fixes it, which will do for now. Might get back to this in July. Thanks @deivid-rodriguez !

[pedrofurtado]

Shopify/bootsnap#168

For me, the workaround is remove the line require 'bootsnap/setup' at config/boot.rb when I need to use byebug. So, I remove this line and restart the server (in my app, it is Puma). When I finish to use byebug, I added again this line.

When the issue (cited by @tzmfreedom) in Ruby Issue Tracking System be finished, this workaround will be not necessary anymore.

[ndbroadbent]

byebug was landing me inside a random method (include? from lib/ruby/2.4.0/set.rb). I'm using Ruby 2.4.3, byebug 10.0.2, bootsnap 1.3.1.

First, I tried deleting the bootsnap cache (rm -rf tmp/cache/bootsnap-*) and stopping spring, but that didn't fix the issue. I also tried calling Bootsnap.setup with compile_cache_iseq: false, but that didn't work either. Commenting out all the call to Bootsnap.setup fixed it. And setting all the bootsnap options to false also fixed it. So after some trial-and-error, I figured out that the problem is with the disable_trace option. This works for me:

  Bootsnap.setup(
    cache_dir:            'tmp/cache',
    development_mode:     true,
    load_path_cache:      true,
    autoload_paths_cache: true,
    disable_trace:        false,
    compile_cache_iseq:   true,
    compile_cache_yaml:   true
  )

I also found this relevant issue on the bootsnap repo.

[rafaelfranca]

This is caused by this bug https://bugs.ruby-lang.org/issues/14702. I asked Koichi if this is a bug that can be fixed and the next 2.5 release and he said he'd work on it. I'm also investigating it myself to see if I can help him with a patch.

[deivid-rodriguez]

Thanks @rafaelfranca, yeah @tzmfreedom discovered and reported this in https://github.com/deivid-rodriguez/byebug/issues/452#issuecomment-389913382. but there was no movement at all on the ticket. I should've pinged Koichi about it, but I forgot to do it, so thanks so much for helping moving this forward :heart:.

[deivid-rodriguez]

Issue has already been fixed in ruby-core, and it'll be backported to ruby 2.5!

[tbolt]

Also experiencing this same issue. Glad to see there is some movement on it.

[amrendrasingh]

I am also facing same issue. Really happy to see work around for this.

[pedrofurtado]

Guys,

This patch was resolved in Ruby 2.5.3. The solution for this issue is to upgrade the Ruby version in projects. 👍

Also for reference: https://github.com/ruby/ruby/commit/c1034574aa7e6f47843b5533411c75d8196b29d8

[deivid-rodriguez]

Thanks for the notice @pedrofurtado!

[buncis]

I'm using rails 5.2.2 and ruby 2.5.3 the problem seems persist

my whole gemfile

source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }

ruby '2.5.3'
gem 'rails', '~> 5.2.2'
gem 'pg', '>= 0.18', '< 2.0'
gem 'puma', '~> 3.11'
gem 'bootsnap', '>= 1.1.0', require: false
gem 'rack-cors'
gem 'graphql'
gem 'devise'
gem 'jwt'

group :development, :test do
  gem 'byebug', platforms: [:mri, :mingw, :x64_mingw]
end

group :development do
  gem 'listen', '>= 3.0.5', '< 3.2'
  gem 'spring'
  gem 'spring-watcher-listen', '~> 2.0.0'
end

my current workaround is removing bootsnap from boot.rb

ENV['BUNDLE_GEMFILE'] ||= File.expand_path('../Gemfile', __dir__)

require 'bundler/setup' # Set up gems listed in the Gemfile.
# require 'bootsnap/setup' # Speed up boot time by caching expensive operations.

similar issue https://github.com/denofevil/debase/issues/66

[augustosamame]

I can confirm that installing ruby 2.5.3 or suggested configuration did not solve the problem for me. Only solution was to remove bootsnap

[andrewjaanuu]

same with Ruby 2.5.3

[pouellet]

I think this is not fixed until 2.5.4 gets released? This seems to be the actual backport that fixed the issue https://bugs.ruby-lang.org/projects/ruby-trunk/repository/ruby_2_5/revisions/66225. Ruby 2.5.3 was released at r65273. Or am I reading this wrong?

[deivid-rodriguez]

Yup, it sounds like we have to wait until 2.5.4. Just to confirm, ruby 2.6.1 is not affected by this, right?

[andrewjaanuu]

tried ruby 2.5.5 and still the same issue

[Lewiscowles1986]

Following bootsnap readme and specifying cache folder worked for me. Byebug / Pry like most debuggers will want you to set to single thread