No connection with DB just after deploy:stop && deploy:start

[noff]

Hello. When do deploy with capistrano, connection with shards lost and I need to wait few minutes before it will be connected again. During this period I receive errors "PG::ConnectionBad: connection is closed".

How to configure Octopus to connect to shards immediately on Rails startup?

[kochka]

Are you using Unicorn ? If yes, maybe you should do this

before_fork do |server, worker|
  if defined?(ActiveRecord::Base)
    ActiveRecord::Base.connection_proxy.instance_variable_get(:@shards).each do |shard, connection_pool|
      connection_pool.disconnect!
    end

    ActiveRecord::Base.connection.disconnect!
  end
end

after_fork do |server, worker|
  if defined?(ActiveRecord::Base)
    ActiveRecord::Base.establish_connection
    ActiveRecord::Base.connection_proxy.instance_variable_get(:@shards).each do |shard, connection_pool| 
      connection_pool.clear_reloadable_connections!
    end
  end
end

[noff]

Solved. Thanks.

[fields]

We saw this problem in production with unicorn, and adding this to before_fork and after_fork did not help. It was not just after a deploy - any time a worker process died (and was restarted by the master), it would throw this same error for a little while after starting.

Removing octopus completely caused the problem to go away. We could not figure out a better fix.

[s12chung]

same problem here..... problem is easily replicated with resque even if this forking code is added.

[s12chung]

for anyone looking, try: #376

[noff]

This does not fix issue. We use it more 4 months.

[ananth99]

This seems to be happening with Phusion Passenger as well. Our Rescue workers are not able to establish a connection with postgresql.

We use a single shard and we use octopus to redirect few actions to fetch data from the shard. I'm unable to reproduce the same issue in our staging env.

This is the error I'm seeing in rescue dashboard: ActiveRecord::StatementInvalid PG::ConnectionBad: connection is closed: SELECT a.attname, format_type(a.atttypid, a.atttypmod), pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod FROM pg_attribute a LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum WHERE a.attrelid = '"users"'::regclass AND a.attnum > 0 AND NOT a.attisdropped ORDER BY a.attnum

Any thoughts on this? TIA!

[biinari]

Cleaner Unicorn config (using public API from Octopus proxy).

before_fork do |server, worker|
  if defined?(ActiveRecord::Base)
    begin
      if defined?(Octopus) && Octopus.enabled?
        ActiveRecord::Base.connection_proxy.clear_all_connections!
      else
        ActiveRecord::Base.connection.disconnect!
      end
    rescue ActiveRecord::ConnectionNotEstablished
      nil
    end
  end
end

after_fork do |server, worker|
  if defined?(ActiveRecord::Base)
    ActiveRecord::Base.establish_connection
    if defined?(Octopus) && Octopus.enabled?
      ActiveRecord::Base.connection_proxy.initialize_shards(Octopus.config)
    end
  end
end