basecamp
Solid Queue only registering Supervisor, not workers/dispatcher/scheduler in Rails 8 / Solid Queue 1.2.1
Describe the bug
In development, running the Solid Queue CLI registers/starts a Supervisor process but the scheduler never loads config/recurring.yml. As a result, solid_queue_recurring_tasks and solid_queue_recurring_executions remain empty and no recurring jobs ever enqueue.
Environment
• macOS Tahoe 26.0.1 (arm64)
• Ruby 3.4.4 (+YJIT)
• Rails 8.0.3
• solid_queue 1.2.1 (bundle info solid_queue confirms)
• DB: Postgres on Neon (separate DB used for Solid Queue)
• SolidQueue::Process.connection.current_database => "atlas_queue"
• Dev environment, running locally.
How I start Solid Queue • Procfile entry used by bin/dev: jobs: bin/jobs (bin/jobs is a thin wrapper: it requires ../config/environment and then SolidQueue::Cli.start(ARGV).) • I also tried launching the CLI directly. See diagnostics below.
Relevant config & env
• .env:
SOLID_QUEUE_RECURRING_SCHEDULE=config/recurring.yml SOLID_QUEUE_CONFIG=config/queue.yml
• config/queue.yml: defines dispatchers/workers; no explicit scheduler block.
• config/recurring.yml: has a development: section with tasks (e.g., get_bookings every 1 minute and a cleanup task).
What I expect
The scheduler should read config/recurring.yml, persist tasks into solid_queue_recurring_tasks / solid_queue_recurring_executions, and enqueue on schedule.
What actually happens • Only the Supervisor starts; no scheduler logs/messages and no recurring rows get created.
Diagnostics collected
1. Env + file visibility (ok):
bin/rails r 'pp ENV.slice("SOLID_QUEUE_CONFIG","SOLID_QUEUE_RECURRING_SCHEDULE","SOLID_QUEUE_SKIP_RECURRING"); puts "Exists(SOLID_QUEUE_CONFIG)? #{File.exist?(ENV["SOLID_QUEUE_CONFIG"].to_s)}"; puts "Exists(SOLID_QUEUE_RECURRING_SCHEDULE)? #{File.exist?(ENV["SOLID_QUEUE_RECURRING_SCHEDULE"].to_s)}"' Output: {"SOLID_QUEUE_CONFIG"=>"config/queue.yml", "SOLID_QUEUE_RECURRING_SCHEDULE"=>"config/recurring.yml"} Exists(SOLID_QUEUE_CONFIG)? true Exists(SOLID_QUEUE_RECURRING_SCHEDULE)? true
2. CLI “about” still boots only Supervisor (no scheduler):
bin/jobs about Logs (abridged): SolidQueue-1.2.1 Register Supervisor ... SolidQueue-1.2.1 Fail claimed jobs ... SolidQueue-1.2.1 Started Supervisor ... (no scheduler or recurring log lines)
3. After the runner is up, recurring tables remain empty:
bin/rails r 'puts "RecurringTask.count=#{SolidQueue::RecurringTask.count}"; puts "RecurringExecution.count=#{SolidQueue::RecurringExecution.count}"; pp SolidQueue::RecurringTask.limit(5).pluck(:key, :queue_name, :schedule); pp SolidQueue::RecurringExecution.order(:run_at).limit(5).pluck(:task_key, :run_at)' Output: RecurringTask.count=0 RecurringExecution.count=0 [] []
Notes • Tables exist with expected columns (e.g., solid_queue_recurring_tasks.key, solid_queue_recurring_executions.task_key). • The same .env is visible to Rails and to the CLI (since bin/jobs requires the Rails environment first). • I also tried setting:
SOLID_QUEUE_LOG_LEVEL=debug
SOLID_QUEUE_CONFIG=config/queue.yml
SOLID_QUEUE_RECURRING_SCHEDULE=config/recurring.yml
bin/jobs start
(If the CLI is supposed to print an explicit “loading recurring schedule” line, I don’t see it.)
Questions
1. In 1.2.1, should the scheduler come up automatically with bin/jobs start when SOLID_QUEUE_RECURRING_SCHEDULE is set? Or is there another env/flag expected?
2. Am I wrong in thinking the supervisor should fork into new processes?
3. Is this issue isolated to this version or environment setup?
Can you copy your config/recurring.yml configuration and your config/queue.yml configuration?
recurring.yml
# examples:
# periodic_cleanup:
# class: CleanSoftDeletedRecordsJob
# queue: background
# args: [ 1000, { batch_size: 500 } ]
# schedule: every hour
# periodic_cleanup_with_command:
# command: "SoftDeletedRecord.due.delete_all"
# priority: 2
# schedule: at 5am every day
production:
clear_solid_queue_finished_jobs:
command: "SolidQueue::Job.clear_finished_in_batches(sleep_between_batches: 0.3)"
schedule: every hour at minute 12
get_bookings:
class: GetBookingsJob
queue: default
schedule: every 1 minutes
development:
get_bookings:
class: GetBookingsJob
queue: default
schedule: every 1 minutes
queue.yml
default: &default
dispatchers:
- polling_interval: 1
batch_size: 500
workers:
- queues: "*"
threads: 3
processes: <%= ENV.fetch("JOB_CONCURRENCY", 1) %>
polling_interval: 0.1
development:
<<: *default
test:
<<: *default
production:
<<: *default
@rosa
I have changed these significantly multiple times with no change in behavior on the forking from the supervisor process.
I should also add that this does work in production and on a collaborator's machine. I am thinking it is a MacOS 26 issue.
Oh, maybe it's the configuration in environments/development.rb? Did you follow all the steps outlined here?
I have the same issue: development does not run recurring tasks, as no scheduler starts. I added the following to config/initializers/solid_queue.rb:
SolidQueue.on_start do |supervisor|
Rails.logger.info "SUPERVISOR START #{supervisor.inspect}"
end
SolidQueue.on_worker_start do |worker|
Rails.logger.info "WORKER START #{worker.inspect}"
end
SolidQueue.on_dispatcher_start do |dispatcher|
Rails.logger.info "DISPATCHER START #{dispatcher.inspect}"
end
SolidQueue.on_scheduler_start do |scheduler|
Rails.logger.info "SCHEDULER START #{scheduler.inspect}"
end
In log/development.log, I do see the SUPERVISOR, WORKER and DISPATCHER lines, but not the SCHEDULER line. Restarting the worker logs the following to STDOUT:
worker | Interrupting...
worker | Exited
worker | Restarting...
worker | Restarted with pid 3847...
worker | SolidQueue-1.2.4 Register Supervisor (28.4ms) pid: 3848, hostname: "iMac.local", process_id: 22, name: "supervisor-75be194e0730b823790a"
worker | SolidQueue-1.2.4 Started Supervisor (44.6ms) pid: 3848, hostname: "iMac.local", process_id: 22, name: "supervisor-75be194e0730b823790a"
worker | SolidQueue-1.2.4 Prune dead processes (7.4ms) size: 0
worker | SolidQueue-1.2.4 Register Dispatcher (20.8ms) pid: 3851, hostname: "iMac.local", process_id: 23, name: "dispatcher-c2b7ad8d596b91f257d4"
worker | SolidQueue-1.2.4 Started Dispatcher (25.2ms) pid: 3851, hostname: "iMac.local", process_id: 23, name: "dispatcher-c2b7ad8d596b91f257d4", polling_interval: 1, batch_size: 500, concurrency_maintenance_interval: 600
worker | SolidQueue-1.2.4 Register Worker (22.8ms) pid: 3852, hostname: "iMac.local", process_id: 24, name: "worker-ca2dccfb15ae70a4f946"
worker | SolidQueue-1.2.4 Started Worker (27.0ms) pid: 3852, hostname: "iMac.local", process_id: 24, name: "worker-ca2dccfb15ae70a4f946", polling_interval: 0.1, queues: "*", thread_pool_size: 1
worker | SolidQueue-1.2.4 Unblock jobs (7.8ms) limit: 500, size: 0
I run env RUBY_DEBUG_OPEN=true bin/jobs through overmind, and my Procfile.dev is:
web: env RUBY_DEBUG_OPEN=true bin/rails server --port=5002
# Run a single SolidQueue supervisor, so we don't have concurrency issues during dev/test
worker: env RUBY_DEBUG_OPEN=true bin/jobs
release: bin/rails db:migrate
config/schedule.yml looks like:
development:
refresh_reminders:
class: RefreshRemindersJob
schedule: */5 * * * *
Yes, I have followed the steps to run in development:
config/database.yml is:
default: &default
adapter: sqlite3
max_connections: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
timeout: 5000
development:
<<: *default
database: storage/development.sqlite3
$ ag solid_queue config/environments
config/environments/development.rb
57: config.active_job.queue_adapter = :solid_queue
And I have a migration that copied queue_schema.rb to the main db migration, and have run this. I can enqueue jobs and they run, but recurring tasks do not, likely because the scheduler process isn't up and running.
This is just another data point.
