resque-scheduler
resque-scheduler copied to clipboard
Use with Redis 5.0 gem causes error loading schedule
After upgrading from Redis gem v4.8.0 to v5.0.2 we got the following error on Heroku.
Our Redis URLs take the form redis://h:[email protected]:12345
Resque.schedule = YAML.load_file(Rails.root.join("config", "resque_schedule.yml"))
RedisClient::CommandError: ERR wrong number of arguments for 'auth' command
Reverting to Redis v4.8.0 corrects the issue.
Seeing some issues with redis 5.x.x gem and BACKGROUND=yes too, scheduler exits silently with 0 code, but background process is not running, from what I gathered in strace -f child process fails with something like:
"NoMethodError: undefined method `reconnect' for #<Redis::Client redis://localhost:6379/0>
I tried the latest version and it still silently fails in background mode (I wish logging wasn't so broken for resque-scheduler). strace shows some errors like:
rake aborted!
Redis::CannotConnectError: getaddrinfo_a: System error
...
Caused by:
RedisClient::CannotConnectError: getaddrinfo_a: System error
/.../shared/bundle/ruby/2.7.0/gems/redis-client-0.9.0/lib/redis_client/ruby_connection.rb:49:in `initialize'
/.../shared/bundle/ruby/2.7.0/gems/redis-client-0.9.0/lib/redis_client.rb:653:in `new'
/.../shared/bundle/ruby/2.7.0/gems/redis-client-0.9.0/lib/redis_client.rb:653:in `block in connect'
/.../shared/bundle/ruby/2.7.0/gems/redis-client-0.9.0/lib/redis_client/middlewares.rb:8:in `connect'
/.../shared/bundle/ruby/2.7.0/gems/redis-client-0.9.0/lib/redis_client.rb:652:in `connect'
/.../shared/bundle/ruby/2.7.0/gems/newrelic_rpm-8.10.1/lib/new_relic/agent/instrumentation/redis/prepend.rb:20:in `block in connect'
/.../shared/bundle/ruby/2.7.0/gems/newrelic_rpm-8.10.1/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:30:in `block in connect_with_tracing'
/.../shared/bundle/ruby/2.7.0/gems/newrelic_rpm-8.10.1/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45:in `block in with_tracing'
/.../shared/bundle/ruby/2.7.0/gems/newrelic_rpm-8.10.1/lib/new_relic/agent/tracer.rb:352:in `capture_segment_error'
/.../shared/bundle/ruby/2.7.0/gems/newrelic_rpm-8.10.1/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45:in `with_tracing'
/.../shared/bundle/ruby/2.7.0/gems/newrelic_rpm-8.10.1/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:30:in `connect_with_tracing'
/.../shared/bundle/ruby/2.7.0/gems/newrelic_rpm-8.10.1/lib/new_relic/agent/instrumentation/redis/prepend.rb:20:in `connect'
/.../shared/bundle/ruby/2.7.0/gems/redis-client-0.9.0/lib/redis_client.rb:646:in `raw_connection'
/.../shared/bundle/ruby/2.7.0/gems/redis-client-0.9.0/lib/redis_client.rb:613:in `ensure_connected'
/.../shared/bundle/ruby/2.7.0/gems/redis-client-0.9.0/lib/redis_client.rb:212:in `call_v'
/.../shared/bundle/ruby/2.7.0/gems/redis-5.0.5/lib/redis/client.rb:73:in `call_v'
/.../shared/bundle/ruby/2.7.0/gems/redis-5.0.5/lib/redis.rb:167:in `block in send_command'
/.../shared/bundle/ruby/2.7.0/gems/redis-5.0.5/lib/redis.rb:166:in `synchronize'
/.../shared/bundle/ruby/2.7.0/gems/redis-5.0.5/lib/redis.rb:166:in `send_command'
/.../shared/bundle/ruby/2.7.0/gems/redis-5.0.5/lib/redis/commands/connection.rb:21:in `ping'
/.../shared/bundle/ruby/2.7.0/gems/redis-namespace-1.9.0/lib/redis/namespace.rb:479:in `call_with_namespace'
/.../shared/bundle/ruby/2.7.0/gems/redis-namespace-1.9.0/lib/redis/namespace.rb:354:in `block (2 levels) in <class:Namespace>'
/.../shared/bundle/ruby/2.7.0/gems/resque-2.4.0/lib/resque/data_store.rb:83:in `reconnect'
/.../shared/bundle/ruby/2.7.0/gems/resque-scheduler-4.7.0/lib/resque/scheduler/env.rb:42:in `setup_backgrounding'
/.../shared/bundle/ruby/2.7.0/gems/resque-scheduler-4.7.0/lib/resque/scheduler/env.rb:17:in `setup'
/.../shared/bundle/ruby/2.7.0/gems/resque-scheduler-4.7.0/lib/resque/scheduler/cli.rb:113:in `setup_env'
/.../shared/bundle/ruby/2.7.0/gems/resque-scheduler-4.7.0/lib/resque/scheduler/tasks.rb:17:in `block (2 levels) in <main>'
/.../shared/bundle/ruby/2.7.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
Tasks: TOP => resque:scheduler
(See full trace by running task with --trace)
After upgrading from Redis gem v4.8.0 to v5.0.2 we got the following error on Heroku.
Our Redis URLs take the form
redis://h:[email protected]:12345Resque.schedule = YAML.load_file(Rails.root.join("config", "resque_schedule.yml")) RedisClient::CommandError: ERR wrong number of arguments for 'auth' commandReverting to Redis v4.8.0 corrects the issue.
I think Redis 5.0 changed the way you specify the server and port. I'm not near a computer though so that may be something you want to look into.
I tried the latest version and it still silently fails in background mode (I wish logging wasn't so broken for
resque-scheduler).straceshows some errors like:rake aborted! Redis::CannotConnectError: getaddrinfo_a: System error ... Caused by: RedisClient::CannotConnectError: getaddrinfo_a: System error /.../shared/bundle/ruby/2.7.0/gems/redis-client-0.9.0/lib/redis_client/ruby_connection.rb:49:in `initialize' /.../shared/bundle/ruby/2.7.0/gems/redis-client-0.9.0/lib/redis_client.rb:653:in `new' /.../shared/bundle/ruby/2.7.0/gems/redis-client-0.9.0/lib/redis_client.rb:653:in `block in connect' /.../shared/bundle/ruby/2.7.0/gems/redis-client-0.9.0/lib/redis_client/middlewares.rb:8:in `connect' /.../shared/bundle/ruby/2.7.0/gems/redis-client-0.9.0/lib/redis_client.rb:652:in `connect' /.../shared/bundle/ruby/2.7.0/gems/newrelic_rpm-8.10.1/lib/new_relic/agent/instrumentation/redis/prepend.rb:20:in `block in connect' /.../shared/bundle/ruby/2.7.0/gems/newrelic_rpm-8.10.1/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:30:in `block in connect_with_tracing' /.../shared/bundle/ruby/2.7.0/gems/newrelic_rpm-8.10.1/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45:in `block in with_tracing' /.../shared/bundle/ruby/2.7.0/gems/newrelic_rpm-8.10.1/lib/new_relic/agent/tracer.rb:352:in `capture_segment_error' /.../shared/bundle/ruby/2.7.0/gems/newrelic_rpm-8.10.1/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:45:in `with_tracing' /.../shared/bundle/ruby/2.7.0/gems/newrelic_rpm-8.10.1/lib/new_relic/agent/instrumentation/redis/instrumentation.rb:30:in `connect_with_tracing' /.../shared/bundle/ruby/2.7.0/gems/newrelic_rpm-8.10.1/lib/new_relic/agent/instrumentation/redis/prepend.rb:20:in `connect' /.../shared/bundle/ruby/2.7.0/gems/redis-client-0.9.0/lib/redis_client.rb:646:in `raw_connection' /.../shared/bundle/ruby/2.7.0/gems/redis-client-0.9.0/lib/redis_client.rb:613:in `ensure_connected' /.../shared/bundle/ruby/2.7.0/gems/redis-client-0.9.0/lib/redis_client.rb:212:in `call_v' /.../shared/bundle/ruby/2.7.0/gems/redis-5.0.5/lib/redis/client.rb:73:in `call_v' /.../shared/bundle/ruby/2.7.0/gems/redis-5.0.5/lib/redis.rb:167:in `block in send_command' /.../shared/bundle/ruby/2.7.0/gems/redis-5.0.5/lib/redis.rb:166:in `synchronize' /.../shared/bundle/ruby/2.7.0/gems/redis-5.0.5/lib/redis.rb:166:in `send_command' /.../shared/bundle/ruby/2.7.0/gems/redis-5.0.5/lib/redis/commands/connection.rb:21:in `ping' /.../shared/bundle/ruby/2.7.0/gems/redis-namespace-1.9.0/lib/redis/namespace.rb:479:in `call_with_namespace' /.../shared/bundle/ruby/2.7.0/gems/redis-namespace-1.9.0/lib/redis/namespace.rb:354:in `block (2 levels) in <class:Namespace>' /.../shared/bundle/ruby/2.7.0/gems/resque-2.4.0/lib/resque/data_store.rb:83:in `reconnect' /.../shared/bundle/ruby/2.7.0/gems/resque-scheduler-4.7.0/lib/resque/scheduler/env.rb:42:in `setup_backgrounding' /.../shared/bundle/ruby/2.7.0/gems/resque-scheduler-4.7.0/lib/resque/scheduler/env.rb:17:in `setup' /.../shared/bundle/ruby/2.7.0/gems/resque-scheduler-4.7.0/lib/resque/scheduler/cli.rb:113:in `setup_env' /.../shared/bundle/ruby/2.7.0/gems/resque-scheduler-4.7.0/lib/resque/scheduler/tasks.rb:17:in `block (2 levels) in <main>' /.../shared/bundle/ruby/2.7.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>' Tasks: TOP => resque:scheduler (See full trace by running task with --trace)
This one is a new one to me. I wish I knew what that system error was. Reverting to redis 4.8 fixes it?
This one is a new one to me. I wish I knew what that system error was. Reverting to redis 4.8 fixes it?
Yes.
FYI I fixed this by changing redis connection string from:
Resque.redis = 'localhost:6379'
to
Resque.redis = '127.0.0.1:6379'
Not sure what's the deal with getaddrinfo_a: System error. It only happens when started by capistrano-redis. I couldn't reproduce it manually.