resque-pool icon indicating copy to clipboard operation
resque-pool copied to clipboard

start without running queues throws error

Open schorsch opened this issue 13 years ago • 8 comments

When starting for the first time without running workers an error is thrown and stops the whole pool from running:

bundle exec resque-pool
=> calls reaper

gems/resque-pool-0.2.0/lib/resque/pool.rb:243:in `reap_all_workers': You have a nil object when you didn't expect it! (NoMethodError)
The error occurred while evaluating nil.queues

Possible solution, add 'if worker' to reaper message:

log "Reaped resque worker[#{status.pid}] (status: #{status.exitstatus}) queues: #{worker.queues.join(",")}" if worker

I am not sure, but i think last week i had it running local without this error. maybe it is the startup time of the workers which prevents the reaper from seeing them.

schorsch avatar Apr 05 '11 18:04 schorsch

Seems to be related to this one: https://github.com/nevans/resque-pool/issues#issue/20

here is some more output, now the manager runs but kills and never restarts workers

bundle exec resque-pool resque-pool-manager[7889]: Resque Pool running in development environment (in /home/georg/trunk) resque-pool-manager[7889]: started manager resque-pool-manager[7889]: Pool contains worker PIDs: [7898, 7896, 7899] resque-pool-worker[7896]: Starting worker minimi:7896:night_jobsresque-pool-worker[7899]: Starting worker minimi:7899:pubsub_publish resque-pool-worker[7898]: Starting worker minimi:7898:scheduler

resque-pool-manager[7889]: WINCH: gracefully stopping all workers resque-pool-manager[7889]: WINCH: gracefully stopping all workers resque-pool-manager[7889]: Reaped resque worker[7899](status: 0) queues: pubsub_publish resque-pool-manager[7889]: Reaped resque worker[7898](status: 0) queues: scheduler resque-pool-manager[7889]: Reaped resque worker[7896](status: 0) queues: night_jobs resque-pool-manager[7889]: WINCH: gracefully stopping all workers resque-pool-manager[7889]: WINCH: gracefully stopping all workers

$ ps -ef f | grep [r]esque georg 7889 2908 3 20:42 pts/0 S+ 0:06 | _ resque-pool-master: managing []

running ubuntu maverik 64bit: ruby 1.8.7 (2010-06-23 patchlevel 299) [x86_64-linux]

schorsch avatar Apr 05 '11 18:04 schorsch

This might be a different issue than your original bug, but something appears to be sending the WINCH signal to your pool manager. Are you doing that manually?

nevans avatar Apr 05 '11 21:04 nevans

no i am just starting the manager without args in dev mode and also deleted my bundle so no old gems are present. I searched in rp gem for WINCH but it is not used anywhere

.. whow this is funny: WINCH get send to the process when i resize my terminal window ???

schorsch avatar Apr 06 '11 21:04 schorsch

WINCH is an abbreviation for window [size] change http://en.wikipedia.org/wiki/SIGWINCH

schorsch avatar Apr 06 '11 21:04 schorsch

Well that explains the WINCH issue (which is different from the original one). I had completely forgotten WINCH's original purpose. :)

I'm copying nginx and Unicorn's signals handling, and WINCH will gracefully shutdown all workers (see "Signals" in the README). You can restart them with HUP.

As to your original bug, yeah we should guard against workers being nil.

nevans avatar Apr 07 '11 19:04 nevans

this issue has been hitting me for while and I have been running resque-pool from inside screen to monitor it :(

jtoy avatar Aug 06 '11 19:08 jtoy

I noticed that this started happening when I upgraded New Relic's rpm_contrib from 1.0.13 to 2.1.4. I haven't looked into it more than that, but yanking rpm_contrib fixed the problem. I'd imagine their resque probes are doing something they shouldn't be.

nirvdrum avatar Oct 12 '11 14:10 nirvdrum

I've been running into this issue since we use resque-pool in development, and as such, our workers are reaped frequently. If it hasn't been fixed, I may take a crack at a pull request.

clifff avatar Aug 15 '12 01:08 clifff