moped icon indicating copy to clipboard operation
moped copied to clipboard

Extreme delays after removing member of a replica set

Open jonhyman opened this issue 11 years ago • 3 comments

Starting yesterday, MongoHQ put some physical servers in my replica set to replace my AWS-backed ones. I added the new servers to my mongoid.yml while they were secondaries, then we cutover to one as the primary. This was all fine.

Today, they removed the AWS backed machines from the replica set. All my apps started responding extremely poorly immediately: we were getting lots of 10 second response times; given that all these requests were a bit over 10 seconds, that seems like some kind of timeout. When I removed the entry from my mongoid.yml for the AWS machines and restarted the servers, response times returned to be normal in the milliseconds.

On Moped 1.4.5 (Mongoid 3.1.2)

jonhyman avatar Apr 06 '13 16:04 jonhyman

this looks like it could also be related to #186

neerfri avatar Apr 26 '13 09:04 neerfri

We face the same problem in our infrastructure. The infrastructure is very dynamic and the ip addresses of the mongodb is changing for all servers from time to time. This is a problem for the current driver, since it caches the ip address of the database (https://github.com/mongoid/moped/blob/master/lib/moped/address.rb#L48). The solution is to allow to set an option, that would disable the caching of the ip so that if the DNS points to a different set of host the connections can be re established. Otherwise the application just times out.

threez avatar May 14 '13 09:05 threez

This is the same issue as #257

michaeldauria avatar Feb 17 '14 16:02 michaeldauria