Redis Sentinel not working
Describe the bug
Latest version(4.2) can't connect to redis using sentinel. Tested in docker setup.
rails-container keeps restarting with error log:
rails-default-1 | + rm -rf /app/tmp/pids/server.pid
rails-default-1 | + rm -rf '/app/tmp/cache/*'
rails-default-1 | + echo 'Waiting for postgres to become ready....'
rails-default-1 | Waiting for postgres to become ready....
rails-default-1 | + docker/entrypoints/helpers/pg_database_url.rb
rails-default-1 | + export 'POSTGRES_PORT=5432'
rails-default-1 | + PG_READY='pg_isready -h postgres-default -p 5432 -U postgres'
rails-default-1 | + pg_isready -h postgres-default -p 5432 -U postgres
rails-default-1 | + echo 'Database ready to accept connections.'
rails-default-1 | + bundle install
rails-default-1 | postgres-default:5432 - accepting connections
rails-default-1 | Database ready to accept connections.
rails-default-1 | Bundle complete! 131 Gemfile dependencies, 260 gems now installed.
rails-default-1 | Gems in the groups 'development' and 'test' were not installed.
rails-default-1 | Bundled gems are installed into /gems
rails-default-1 | 1 installed gem you directly depend on is looking for funding.
rails-default-1 | Run bundle fund for details
rails-default-1 | + BUNDLE='bundle check'
rails-default-1 | + bundle check
rails-default-1 | The Gemfile's dependencies are satisfied
rails-default-1 | + exec bundle exec rails s -p 3000 -b 0.0.0.0
rails-default-1 | => Booting Puma
rails-default-1 | => Rails 7.0.8.7 application starting in production
rails-default-1 | => Run bin/rails server --help for more startup options
rails-default-1 | Exiting
rails-default-1 | /gems/ruby/3.3.0/gems/redis-client-0.21.1/lib/redis_client/sentinel_config.rb:146:in block in resolve_master': undefined method call' for an instance of Redis::Client (NoMethodError)
rails-default-1 |
rails-default-1 | host, port = sentinel_client.call("SENTINEL", "get-master-addr-by-name", @name)
rails-default-1 | ^^^^^
rails-default-1 | Did you mean? call_v
rails-default-1 | from /gems/ruby/3.3.0/gems/redis-client-0.21.1/lib/redis_client/sentinel_config.rb:188:in block in each_sentinel' rails-default-1 | from /gems/ruby/3.3.0/gems/redis-client-0.21.1/lib/redis_client/sentinel_config.rb:184:in each'
rails-default-1 | from /gems/ruby/3.3.0/gems/redis-client-0.21.1/lib/redis_client/sentinel_config.rb:184:in each_sentinel' rails-default-1 | from /gems/ruby/3.3.0/gems/redis-client-0.21.1/lib/redis_client/sentinel_config.rb:145:in resolve_master'
rails-default-1 | from /gems/ruby/3.3.0/gems/redis-client-0.21.1/lib/redis_client/sentinel_config.rb:137:in block in config' rails-default-1 | from /gems/ruby/3.3.0/gems/redis-client-0.21.1/lib/redis_client/sentinel_config.rb:135:in synchronize'
rails-default-1 | from /gems/ruby/3.3.0/gems/redis-client-0.21.1/lib/redis_client/sentinel_config.rb:135:in config' rails-default-1 | from /gems/ruby/3.3.0/gems/redis-client-0.21.1/lib/redis_client/sentinel_config.rb:81:in host'
rails-default-1 | from /gems/ruby/3.3.0/gems/redis-client-0.21.1/lib/redis_client/ruby_connection.rb:119:in connect' rails-default-1 | from /gems/ruby/3.3.0/gems/redis-client-0.21.1/lib/redis_client/ruby_connection.rb:51:in initialize'
rails-default-1 | from /gems/ruby/3.3.0/gems/redis-client-0.21.1/lib/redis_client.rb:746:in new' rails-default-1 | from /gems/ruby/3.3.0/gems/redis-client-0.21.1/lib/redis_client.rb:746:in block in connect'
rails-default-1 | from /gems/ruby/3.3.0/gems/redis-client-0.21.1/lib/redis_client/middlewares.rb:12:in connect' rails-default-1 | from /gems/ruby/3.3.0/gems/redis-client-0.21.1/lib/redis_client.rb:745:in connect'
rails-default-1 | from /gems/ruby/3.3.0/gems/redis-client-0.21.1/lib/redis_client.rb:732:in raw_connection' rails-default-1 | from /gems/ruby/3.3.0/gems/redis-client-0.21.1/lib/redis_client.rb:697:in ensure_connected'
rails-default-1 | from /gems/ruby/3.3.0/gems/redis-client-0.21.1/lib/redis_client.rb:292:in call_v' rails-default-1 | from /gems/ruby/3.3.0/gems/redis-5.0.6/lib/redis/client.rb:73:in call_v'
rails-default-1 | from /gems/ruby/3.3.0/gems/redis-5.0.6/lib/redis.rb:167:in block in send_command' rails-default-1 | from /gems/ruby/3.3.0/gems/redis-5.0.6/lib/redis.rb:166:in synchronize'
rails-default-1 | from /gems/ruby/3.3.0/gems/redis-5.0.6/lib/redis.rb:166:in send_command' rails-default-1 | from /gems/ruby/3.3.0/gems/redis-5.0.6/lib/redis/commands/strings.rb:191:in get'
rails-default-1 | from /gems/ruby/3.3.0/gems/redis-namespace-1.10.0/lib/redis/namespace.rb:558:in wrapped_send' rails-default-1 | from /gems/ruby/3.3.0/gems/redis-namespace-1.10.0/lib/redis/namespace.rb:515:in call_with_namespace'
rails-default-1 | from /gems/ruby/3.3.0/gems/redis-namespace-1.10.0/lib/redis/namespace.rb:389:in block (2 levels) in <class:Namespace>' rails-default-1 | from /app/lib/global_config.rb:42:in block in load_from_cache'
rails-default-1 | from /gems/ruby/3.3.0/gems/connection_pool-2.4.1/lib/connection_pool.rb:110:in block (2 levels) in with' rails-default-1 | from /gems/ruby/3.3.0/gems/connection_pool-2.4.1/lib/connection_pool.rb:109:in handle_interrupt'
rails-default-1 | from /gems/ruby/3.3.0/gems/connection_pool-2.4.1/lib/connection_pool.rb:109:in block in with' rails-default-1 | from /gems/ruby/3.3.0/gems/connection_pool-2.4.1/lib/connection_pool.rb:106:in handle_interrupt'
rails-default-1 | from /gems/ruby/3.3.0/gems/connection_pool-2.4.1/lib/connection_pool.rb:106:in with' rails-default-1 | from /app/lib/global_config.rb:42:in load_from_cache'
rails-default-1 | from /app/lib/global_config.rb:20:in get_value' rails-default-1 | from /app/lib/chatwoot_app.rb:21:in chatwoot_cloud?'
rails-default-1 | from /app/app/dashboards/account_dashboard.rb:17:in <class:AccountDashboard>' rails-default-1 | from /app/app/dashboards/account_dashboard.rb:3:in require' rails-default-1 | from /usr/local/lib/ruby/3.3.0/bundled_gems.rb:74:in block (2 levels) in replace_require'
rails-default-1 | from /gems/ruby/3.3.0/gems/bootsnap-1.16.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in require' rails-default-1 | from /gems/ruby/3.3.0/gems/zeitwerk-2.6.17/lib/zeitwerk/kernel.rb:26:in require'
rails-default-1 | from /gems/ruby/3.3.0/gems/zeitwerk-2.6.17/lib/zeitwerk/cref.rb:91:in const_get' rails-default-1 | from /gems/ruby/3.3.0/gems/zeitwerk-2.6.17/lib/zeitwerk/cref.rb:91:in get'
rails-default-1 | from /gems/ruby/3.3.0/gems/zeitwerk-2.6.17/lib/zeitwerk/loader/eager_load.rb:173:in block in actual_eager_load_dir' rails-default-1 | from /gems/ruby/3.3.0/gems/zeitwerk-2.6.17/lib/zeitwerk/loader/helpers.rb:47:in block in ls'
rails-default-1 | from /gems/ruby/3.3.0/gems/zeitwerk-2.6.17/lib/zeitwerk/loader/helpers.rb:25:in each' rails-default-1 | from /gems/ruby/3.3.0/gems/zeitwerk-2.6.17/lib/zeitwerk/loader/helpers.rb:25:in ls'
rails-default-1 | from /gems/ruby/3.3.0/gems/zeitwerk-2.6.17/lib/zeitwerk/loader/eager_load.rb:168:in actual_eager_load_dir' rails-default-1 | from /gems/ruby/3.3.0/gems/zeitwerk-2.6.17/lib/zeitwerk/loader/eager_load.rb:17:in block (2 levels) in eager_load'
rails-default-1 | from /gems/ruby/3.3.0/gems/zeitwerk-2.6.17/lib/zeitwerk/loader/eager_load.rb:16:in each' rails-default-1 | from /gems/ruby/3.3.0/gems/zeitwerk-2.6.17/lib/zeitwerk/loader/eager_load.rb:16:in block in eager_load'
rails-default-1 | from /gems/ruby/3.3.0/gems/zeitwerk-2.6.17/lib/zeitwerk/loader/eager_load.rb:10:in synchronize' rails-default-1 | from /gems/ruby/3.3.0/gems/zeitwerk-2.6.17/lib/zeitwerk/loader/eager_load.rb:10:in eager_load'
rails-default-1 | from /gems/ruby/3.3.0/gems/zeitwerk-2.6.17/lib/zeitwerk/loader.rb:413:in block in eager_load_all' rails-default-1 | from /gems/ruby/3.3.0/gems/zeitwerk-2.6.17/lib/zeitwerk/loader.rb:411:in each'
rails-default-1 | from /gems/ruby/3.3.0/gems/zeitwerk-2.6.17/lib/zeitwerk/loader.rb:411:in eager_load_all' rails-default-1 | from /gems/ruby/3.3.0/gems/railties-7.0.8.7/lib/rails/application/finisher.rb:74:in block in module:Finisher'
rails-default-1 | from /gems/ruby/3.3.0/gems/railties-7.0.8.7/lib/rails/initializable.rb:32:in instance_exec' rails-default-1 | from /gems/ruby/3.3.0/gems/railties-7.0.8.7/lib/rails/initializable.rb:32:in run'
rails-default-1 | from /gems/ruby/3.3.0/gems/railties-7.0.8.7/lib/rails/initializable.rb:61:in block in run_initializers' rails-default-1 | from /usr/local/lib/ruby/3.3.0/tsort.rb:231:in block in tsort_each'
rails-default-1 | from /usr/local/lib/ruby/3.3.0/tsort.rb:353:in block (2 levels) in each_strongly_connected_component' rails-default-1 | from /usr/local/lib/ruby/3.3.0/tsort.rb:434:in each_strongly_connected_component_from'
rails-default-1 | from /usr/local/lib/ruby/3.3.0/tsort.rb:352:in block in each_strongly_connected_component' rails-default-1 | from /usr/local/lib/ruby/3.3.0/tsort.rb:350:in each'
rails-default-1 | from /usr/local/lib/ruby/3.3.0/tsort.rb:350:in call' rails-default-1 | from /usr/local/lib/ruby/3.3.0/tsort.rb:350:in each_strongly_connected_component'
rails-default-1 | from /usr/local/lib/ruby/3.3.0/tsort.rb:229:in tsort_each' rails-default-1 | from /usr/local/lib/ruby/3.3.0/tsort.rb:208:in tsort_each'
rails-default-1 | from /gems/ruby/3.3.0/gems/railties-7.0.8.7/lib/rails/initializable.rb:60:in run_initializers' rails-default-1 | from /gems/ruby/3.3.0/gems/railties-7.0.8.7/lib/rails/application.rb:372:in initialize!'
rails-default-1 | from /app/config/environment.rb:5:in <main>' rails-default-1 | from config.ru:3:in require_relative'
rails-default-1 | from config.ru:3:in block in <main>' rails-default-1 | from /gems/ruby/3.3.0/gems/rack-2.2.14/lib/rack/builder.rb:116:in eval'
rails-default-1 | from /gems/ruby/3.3.0/gems/rack-2.2.14/lib/rack/builder.rb:116:in new_from_string' rails-default-1 | from /gems/ruby/3.3.0/gems/rack-2.2.14/lib/rack/builder.rb:105:in load_file'
rails-default-1 | from /gems/ruby/3.3.0/gems/rack-2.2.14/lib/rack/builder.rb:66:in parse_file' rails-default-1 | from /gems/ruby/3.3.0/gems/rack-2.2.14/lib/rack/server.rb:349:in build_app_and_options_from_config'
rails-default-1 | from /gems/ruby/3.3.0/gems/rack-2.2.14/lib/rack/server.rb:249:in app' rails-default-1 | from /gems/ruby/3.3.0/gems/rack-2.2.14/lib/rack/server.rb:422:in wrapped_app'
rails-default-1 | from /gems/ruby/3.3.0/gems/rack-2.2.14/lib/rack/server.rb:312:in block in start' rails-default-1 | from /gems/ruby/3.3.0/gems/rack-2.2.14/lib/rack/server.rb:379:in handle_profiling'
rails-default-1 | from /gems/ruby/3.3.0/gems/rack-2.2.14/lib/rack/server.rb:311:in start' rails-default-1 | from /gems/ruby/3.3.0/gems/railties-7.0.8.7/lib/rails/commands/server/server_command.rb:38:in start'
rails-default-1 | from /gems/ruby/3.3.0/gems/railties-7.0.8.7/lib/rails/commands/server/server_command.rb:143:in block in perform' rails-default-1 | from <internal:kernel>:90:in tap'
rails-default-1 | from /gems/ruby/3.3.0/gems/railties-7.0.8.7/lib/rails/commands/server/server_command.rb:134:in perform' rails-default-1 | from /gems/ruby/3.3.0/gems/thor-1.3.1/lib/thor/command.rb:28:in run'
rails-default-1 | from /gems/ruby/3.3.0/gems/thor-1.3.1/lib/thor/invocation.rb:127:in invoke_command' rails-default-1 | from /gems/ruby/3.3.0/gems/thor-1.3.1/lib/thor.rb:527:in dispatch'
rails-default-1 | from /gems/ruby/3.3.0/gems/railties-7.0.8.7/lib/rails/command/base.rb:87:in perform' rails-default-1 | from /gems/ruby/3.3.0/gems/railties-7.0.8.7/lib/rails/command.rb:48:in invoke'
rails-default-1 | from /gems/ruby/3.3.0/gems/railties-7.0.8.7/lib/rails/commands.rb:18:in <main>' rails-default-1 | from /usr/local/lib/ruby/3.3.0/bundled_gems.rb:74:in require'
rails-default-1 | from /usr/local/lib/ruby/3.3.0/bundled_gems.rb:74:in block (2 levels) in replace_require' rails-default-1 | from /gems/ruby/3.3.0/gems/bootsnap-1.16.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in require'
rails-default-1 | from bin/rails:4:in `
rails-default-1 exited with code 0
To Reproduce
-
Install Docker on your VM
-
edit .env to use redis sentinel parameters: REDIS_SENTINELS=ip1:26379,ip2:26380,ip3:26381 REDIS_SENTINEL_MASTER_NAME=mymaster REDIS_PASSWORD=redis REDIS_SENTINEL_PASSWORD=redis
-
Prepare the database docker compose run --rm rails bundle exec rails db:chatwoot_prepare
-
build and start the service docker compose up -d --build
Expected behavior
app starts and works with redis sentinel without errors
Environment
Docker
Cloud Provider
None
Platform
Browser
Operating system
No response
Browser and version
No response
Docker (if applicable)
No response
Additional context
No response
The key error causing your Rails container to crash and restart repeatedly is:
NoMethodError: undefined method call' for an instance of Redis::Client
redis-client 0.22.x changed the API In recent versions of redis-client, Redis::Client no longer responds to .call. Instead, you must use .call_v
This means some code is calling the Redis Sentinel client with the old call(...) method, which no longer exists.
So we need to change it to call_v
I have the same problem and I don't know how to solve it.
REDIS_URL=redis://mymaster REDIS_PASSWORD=Abc123 REDIS_SENTINELS=redis-sentinel-01:26379,redis-sentinel-02:26379,redis-sentinel-03:26379 REDIS_SENTINEL_MASTER_NAME=mymaster REDIS_SENTINEL_PASSWORD=Abc123
Any luck solving this????
Any luck solving this????
No. Use Redis Standalone.