solid_queue icon indicating copy to clipboard operation
solid_queue copied to clipboard

See ActiveSupport error after upgrading to 0.3.0: undefined local variable or method `__callbacks' for class ActiveSupport::CurrentAttributes

Open wheee opened this issue 11 months ago • 5 comments

Hi there,

Note that I have SolidQueue already installed in my project, so upgraded to 0.3.0 which installed fugit. After running the migrations to add the new table/indices, starting up my foreman resulted in the following error:

08:54:49 solid_queue.1 | [SolidQueue] Starting Dispatcher(pid=13811, hostname=Jamess-Mac-Studio.local, metadata={:polling_interval=>0.1, :batch_size=>500, :concurrency_maintenance_interval=>300, :recurring_schedule=>nil})
08:54:49 solid_queue.1 | [SolidQueue] Starting Worker(pid=13812, hostname=Jamess-Mac-Studio.local, metadata={:polling_interval=>0.1, :queues=>"recurring", :thread_pool_size=>1})
08:54:49 solid_queue.1 | [SolidQueue] Starting Worker(pid=13813, hostname=Jamess-Mac-Studio.local, metadata={:polling_interval=>0.1, :queues=>"recurring", :thread_pool_size=>1})
08:54:49 solid_queue.1 | [SolidQueue] Starting Worker(pid=13814, hostname=Jamess-Mac-Studio.local, metadata={:polling_interval=>0.1, :queues=>"render", :thread_pool_size=>1})
08:54:49 solid_queue.1 | [SolidQueue] Starting Worker(pid=13815, hostname=Jamess-Mac-Studio.local, metadata={:polling_interval=>0.1, :queues=>"operations,simulations", :thread_pool_size=>1})
08:54:49 solid_queue.1 | [SolidQueue] Starting Worker(pid=13816, hostname=Jamess-Mac-Studio.local, metadata={:polling_interval=>0.1, :queues=>"operations,simulations", :thread_pool_size=>1})
08:54:49 solid_queue.1 | [SolidQueue] Starting Worker(pid=13817, hostname=Jamess-Mac-Studio.local, metadata={:polling_interval=>0.1, :queues=>"operations,simulations", :thread_pool_size=>1})
08:54:49 solid_queue.1 | [SolidQueue] Starting Worker(pid=13818, hostname=Jamess-Mac-Studio.local, metadata={:polling_interval=>0.1, :queues=>"operations,simulations", :thread_pool_size=>1})
08:54:49 solid_queue.1 | [SolidQueue] Starting Worker(pid=13819, hostname=Jamess-Mac-Studio.local, metadata={:polling_interval=>0.1, :queues=>"operations,simulations", :thread_pool_size=>1})
08:54:49 solid_queue.1 | [SolidQueue] Starting Worker(pid=13821, hostname=Jamess-Mac-Studio.local, metadata={:polling_interval=>0.1, :queues=>"processors", :thread_pool_size=>1})
08:54:49 solid_queue.1 | [SolidQueue] Starting Worker(pid=13823, hostname=Jamess-Mac-Studio.local, metadata={:polling_interval=>0.1, :queues=>"processors", :thread_pool_size=>1})
08:54:49 solid_queue.1 | [SolidQueue] Starting Worker(pid=13825, hostname=Jamess-Mac-Studio.local, metadata={:polling_interval=>0.1, :queues=>"processors", :thread_pool_size=>1})
08:54:49 solid_queue.1 | [SolidQueue] Starting Worker(pid=13828, hostname=Jamess-Mac-Studio.local, metadata={:polling_interval=>0.1, :queues=>"streamers", :thread_pool_size=>1})
08:54:49 solid_queue.1 | [SolidQueue] Starting Worker(pid=13832, hostname=Jamess-Mac-Studio.local, metadata={:polling_interval=>0.1, :queues=>"streamers", :thread_pool_size=>1})
08:54:49 solid_queue.1 | [SolidQueue] Starting Worker(pid=13838, hostname=Jamess-Mac-Studio.local, metadata={:polling_interval=>0.1, :queues=>"streamers", :thread_pool_size=>1})
08:54:49 solid_queue.1 | [SolidQueue] Starting Worker(pid=13840, hostname=Jamess-Mac-Studio.local, metadata={:polling_interval=>0.1, :queues=>"streamers", :thread_pool_size=>1})
08:54:49 solid_queue.1 | [SolidQueue] Starting Worker(pid=13842, hostname=Jamess-Mac-Studio.local, metadata={:polling_interval=>0.1, :queues=>"streamers", :thread_pool_size=>1})
08:54:49 solid_queue.1 | /Users/.../gems/activesupport-7.1.3.2/lib/active_support/current_attributes.rb:95: warning: already initialized constant ActiveSupport::CurrentAttributes::INVALID_ATTRIBUTE_NAMES
08:54:49 solid_queue.1 | /Users/.../gems/activesupport-7.1.3.2/lib/active_support/current_attributes.rb:95: warning: previous definition of INVALID_ATTRIBUTE_NAMES was here
08:54:49 solid_queue.1 | /Users/.../gems/activesupport-7.1.3.2/lib/active_support/current_attributes.rb:95: warning: already initialized constant ActiveSupport::CurrentAttributes::INVALID_ATTRIBUTE_NAMES
08:54:49 solid_queue.1 | /Users/.../gems/activesupport-7.1.3.2/lib/active_support/current_attributes.rb:95: warning: previous definition of INVALID_ATTRIBUTE_NAMES was here
08:54:49 solid_queue.1 | /Users/.../gems/activesupport-7.1.3.2/lib/active_support/current_attributes.rb:95: warning: already initialized constant ActiveSupport::CurrentAttributes::INVALID_ATTRIBUTE_NAMES
08:54:49 solid_queue.1 | /Users/.../gems/activesupport-7.1.3.2/lib/active_support/current_attributes.rb:95: warning: previous definition of INVALID_ATTRIBUTE_NAMES was here
08:54:49 solid_queue.1 | /Users/.../gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:977:in `set_callbacks': undefined local variable or method `__callbacks' for class ActiveSupport::CurrentAttributes (NameError)
08:54:49 solid_queue.1 |
08:54:49 solid_queue.1 |               self.__callbacks = __callbacks.dup
08:54:49 solid_queue.1 |                                  ^^^^^^^^^^^
08:54:49 solid_queue.1 | Did you mean?  callbacks
08:54:49 solid_queue.1 |                set_callbacks
08:54:49 solid_queue.1 |                get_callbacks
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:947:in `block (2 levels) in define_callbacks'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:946:in `each'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:946:in `block in define_callbacks'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:943:in `each'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:943:in `define_callbacks'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/current_attributes.rb:93:in `<class:CurrentAttributes>'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/current_attributes.rb:91:in `<module:ActiveSupport>'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/current_attributes.rb:7:in `<main>'
08:54:49 solid_queue.1 | 	from <internal:/Users/.../.rbenv/versions/3.3.0/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
08:54:49 solid_queue.1 | 	from <internal:/Users/.../.rbenv/versions/3.3.0/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
08:54:49 solid_queue.1 | 	from /Users/.../gems/bootsnap-1.18.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
08:54:49 solid_queue.1 | 	from /Users/.../gems/zeitwerk-2.6.13/lib/zeitwerk/kernel.rb:34:in `require'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/railtie.rb:50:in `block (2 levels) in <class:Railtie>'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:448:in `instance_exec'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:448:in `block in make_lambda'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:202:in `block (2 levels) in halting'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:707:in `block (2 levels) in default_terminator'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:706:in `catch'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:706:in `block in default_terminator'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:203:in `block in halting'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:598:in `block in invoke_before'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:598:in `each'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:598:in `invoke_before'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:109:in `run_callbacks'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/execution_wrapper.rb:143:in `complete'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/execution_wrapper.rb:137:in `complete!'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/execution_wrapper.rb:81:in `ensure in block in run!'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/execution_wrapper.rb:81:in `block in run!'
08:54:49 solid_queue.1 | 	from <internal:kernel>:90:in `tap'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/execution_wrapper.rb:75:in `run!'
08:54:49 solid_queue.1 | 	from /Users/.../gems/activesupport-7.1.3.2/lib/active_support/execution_wrapper.rb:90:in `wrap'
08:54:49 solid_queue.1 | 	from /Users/.../gems/solid_queue-0.3.0/lib/solid_queue/app_executor.rb:7:in `wrap_in_app_executor'
08:54:49 solid_queue.1 | 	from /Users/.../gems/solid_queue-0.3.0/lib/solid_queue/processes/poller.rb:26:in `block in start_loop'
08:54:49 solid_queue.1 | 	from <internal:kernel>:187:in `loop'
08:54:49 solid_queue.1 | 	from /Users/.../gems/solid_queue-0.3.0/lib/solid_queue/processes/poller.rb:23:in `start_loop'
08:54:49 solid_queue.1 | 	from /Users/.../gems/solid_queue-0.3.0/lib/solid_queue/processes/poller.rb:18:in `run'
08:54:49 solid_queue.1 | 	from /Users/.../gems/solid_queue-0.3.0/lib/solid_queue/processes/runnable.rb:13:in `start'
08:54:49 solid_queue.1 | 	from /Users/.../gems/solid_queue-0.3.0/lib/solid_queue/supervisor.rb:123:in `block in start_fork'
08:54:49 solid_queue.1 | 	from /Users/.../gems/solid_queue-0.3.0/lib/solid_queue/supervisor.rb:122:in `fork'
08:54:49 solid_queue.1 | 	from /Users/.../gems/solid_queue-0.3.0/lib/solid_queue/supervisor.rb:122:in `start_fork'
08:54:49 solid_queue.1 | 	from /Users/.../gems/solid_queue-0.3.0/lib/solid_queue/supervisor.rb:72:in `block in start_forks'
08:54:49 solid_queue.1 | 	from /Users/.../gems/solid_queue-0.3.0/lib/solid_queue/supervisor.rb:72:in `each'
08:54:49 solid_queue.1 | 	from /Users/.../gems/solid_queue-0.3.0/lib/solid_queue/supervisor.rb:72:in `start_forks'
08:54:49 solid_queue.1 | 	from /Users/.../gems/solid_queue-0.3.0/lib/solid_queue/supervisor.rb:45:in `supervise'
08:54:49 solid_queue.1 | 	from /Users/.../gems/solid_queue-0.3.0/lib/solid_queue/supervisor.rb:26:in `start'
08:54:49 solid_queue.1 | 	from /Users/.../gems/solid_queue-0.3.0/lib/solid_queue/supervisor.rb:14:in `start'
08:54:49 solid_queue.1 | 	from /Users/.../gems/solid_queue-0.3.0/lib/solid_queue/tasks.rb:4:in `block (2 levels) in <main>'
08:54:49 solid_queue.1 | 	from /Users/.../.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `block in execute'
08:54:49 solid_queue.1 | 	from /Users/.../.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `each'
08:54:49 solid_queue.1 | 	from /Users/.../.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `execute'
08:54:49 solid_queue.1 | 	from /Users/.../.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
08:54:49 solid_queue.1 | 	from /Users/.../.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:199:in `synchronize'
08:54:49 solid_queue.1 | 	from /Users/.../.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:199:in `invoke_with_call_chain'
08:54:49 solid_queue.1 | 	from /Users/.../.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:188:in `invoke'
08:54:49 solid_queue.1 | 	from /Users/.../.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:182:in `invoke_task'
08:54:49 solid_queue.1 | 	from /Users/.../.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:138:in `block (2 levels) in top_level'
08:54:49 solid_queue.1 | 	from /Users/.../.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:138:in `each'
08:54:49 solid_queue.1 | 	from /Users/.../.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:138:in `block in top_level'
08:54:49 solid_queue.1 | 	from /Users/.../.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:147:in `run_with_threads'
08:54:49 solid_queue.1 | 	from /Users/.../.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:132:in `top_level'
08:54:49 solid_queue.1 | 	from /Users/.../.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:83:in `block in run'
08:54:49 solid_queue.1 | 	from /Users/.../.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:208:in `standard_exception_handling'
08:54:49 solid_queue.1 | 	from /Users/.../.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:80:in `run'
08:54:49 solid_queue.1 | 	from /Users/.../.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/exe/rake:27:in `<top (required)>'
08:54:49 solid_queue.1 | 	from /Users/.../.rbenv/versions/3.3.0/bin/rake:25:in `load'
08:54:49 solid_queue.1 | 	from /Users/.../.rbenv/versions/3.3.0/bin/rake:25:in `<top (required)>'
08:54:49 solid_queue.1 | 	from /Users/.../gems/bundler-2.4.6/lib/bundler/cli/exec.rb:58:in `load'
08:54:49 solid_queue.1 | 	from /Users/.../gems/bundler-2.4.6/lib/bundler/cli/exec.rb:58:in `kernel_load'
08:54:49 solid_queue.1 | 	from /Users/.../gems/bundler-2.4.6/lib/bundler/cli/exec.rb:23:in `run'
08:54:49 solid_queue.1 | 	from /Users/.../gems/bundler-2.4.6/lib/bundler/cli.rb:491:in `exec'
08:54:49 solid_queue.1 | 	from /Users/.../gems/bundler-2.4.6/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
08:54:49 solid_queue.1 | 	from /Users/.../gems/bundler-2.4.6/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
08:54:49 solid_queue.1 | 	from /Users/.../gems/bundler-2.4.6/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
08:54:49 solid_queue.1 | 	from /Users/.../gems/bundler-2.4.6/lib/bundler/cli.rb:34:in `dispatch'
08:54:49 solid_queue.1 | 	from /Users/.../gems/bundler-2.4.6/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
08:54:49 solid_queue.1 | 	from /Users/.../gems/bundler-2.4.6/lib/bundler/cli.rb:28:in `start'
08:54:49 solid_queue.1 | 	from /Users/.../gems/bundler-2.4.6/exe/bundle:45:in `block in <top (required)>'
08:54:49 solid_queue.1 | 	from /Users/.../gems/bundler-2.4.6/lib/bundler/friendly_errors.rb:117:in `with_friendly_errors'
08:54:49 solid_queue.1 | 	from /Users/.../gems/bundler-2.4.6/exe/bundle:33:in `<top (required)>'
08:54:49 solid_queue.1 | 	from /Users/.../bin/bundle:25:in `load'
08:54:49 solid_queue.1 | 	from /Users/.../bin/bundle:25:in `<main>'
08:54:49 solid_queue.1 | /Users/.../gems/activesupport-7.1.3.2/lib/active_support/callbacks.rb:977:in `set_callbacks': undefined local variable or method `__callbacks' for class ActiveSupport::CurrentAttributes (NameError)`

In terms of impact, my regular jobs are still being processed, but recurring jobs don't seem to be work, but haven't dug in too deep.

Would appreciate some support on this, thanks!

wheee avatar Mar 27 '24 13:03 wheee

Hey @wheee, sorry for the delay! This one is quite strange, haven't seen any error like this before 🤔 Would you mind sharing your solid_queue.yml configuration?

rosa avatar Apr 01 '24 18:04 rosa

Not the original submitter, but I just had this same issue happen to me, and (at least in my case) it was caused by spacing issues in the config/solid_queue.yml file. At least in VSCode, when I bulk uncommented the example yaml file provided by this gem, only a single space was added in front of each of the environment sections - for example:

default: &default
  dispatchers:
    # dispatcher config here
  workers:
    # worker config here

development:
 <<: *default

test:
 <<: *default

production:
 <<: *default

(hard to see in the editor and very subtle). Ensuring two spaces preceded each << fixed this particular issue in my case.

bananastalktome avatar Apr 04 '24 19:04 bananastalktome

FWIW I also get this happening from time to time, but my config file looks OK:

default: &default
  dispatchers:
    - polling_interval: 1
      batch_size: 500
      recurring_tasks:
        summarize_daily_entries_job:
          class: SummarizeDailyEntriesJob
          args: []
          schedule: every hour
  workers:
    - queues: "default"
      threads: 3
      processes: 1
      polling_interval: 0.1
    - queues: "low"
      threads: 1
      processes: 1
      polling_interval: 1

development:
  <<: *default

test:
  <<: *default

production:
  <<: *default

EDIT: I'm using Overmind for process management, Mac OS Sonomo 14.4.1.

jmdfm avatar Apr 10 '24 22:04 jmdfm

Sorry @rosa, have been busy myself as well. I'll definitely try over the weekend to reproduce it.

But given what @bananastalktome said, it sounds about right.

When I tried to upgrade the first time, I definitely added some recurring_tasks and perhaps I added an extra space somewhere (usually my IDE would trim it, but perhaps it's not doing it for yml files).

Will report back later, and thanks @bananastalktome!

wheee avatar Apr 12 '24 14:04 wheee

I haven't had time to really investigate this, but I've had the same issue, and managed to narrow down the cause in my app.

I had a Rake wrapper task that check/ensures that regular tasks are running when the worker starts. Simplified example:

namespace :job_wrapper do

  desc 'Start SolidQueue and ensure regular tasks are kicked off'
  task start_with_regulars: :environment do

    # Schedule MyJob if not already scheduled
    unless Rails.cache.read(MyJob::KEY)
      MyJob.perform_now
    end

    # Start the SolidQueue
    Rake::Task['solid_queue:start'].invoke
  end

end

I narrowed down the issue to the perform_now line. If I comment it out or switch it to perform_later, the __callbacks error is not longer raised. Hopefully that helps someone trace down the root cause or make changes in their apps to suit.

justinfrench avatar Apr 14 '24 01:04 justinfrench

Finally got around to upgrading solid_queue and I'm able able to use the recurring jobs without any issue. Pretty sure it was due to an extra space in my yaml file (at the time).

Thanks!

wheee avatar Jun 30 '24 14:06 wheee