Dropping an Index on a Replica Set

[simontonsoftware]

Using Indexes#drop on my development machine works fine, but running the same code on our staging or production environments, which use a replica set, causes the following error. Is this a bug, or is there another way to drop indexes I haven't found?

D, [2014-12-08T19:33:40.804920 #2] DEBUG -- :   MOPED: <snip> QUERY        database=<snip> collection=system.indexes selector={:ns=>"<snip>", :key=>{<snip>}} flags=[:slave_ok] limit=-1 skip=0 batch_size=nil fields=nil runtime: 1.7464ms
D, [2014-12-08T19:33:40.811542 #2] DEBUG -- :   MOPED: <snip> COMMAND      database=<snip> command={:deleteIndexes=>"<snip>", :index=>"<snip>"} runtime: 6.0027ms
Moped::Errors::OperationFailure: The operation: #<Moped::Protocol::Command
  @length=112
  @request_id=8
  @response_to=0
  @op_code=2004
  @flags=[:slave_ok]
  @full_collection_name="<snip>.$cmd"
  @skip=0
  @limit=-1
  @selector={:deleteIndexes=>"<snip>", :index=>"<snip>"}
  @fields=nil>
failed with error "not master"
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/operation/read.rb:50:in `block in execute'
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/node.rb:594:in `[]'
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/node.rb:594:in `block (2 levels) in flush'
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/node.rb:593:in `map'
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/node.rb:593:in `block in flush'
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/node.rb:617:in `block in logging'
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/instrumentable/log.rb:26:in `instrument'
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/instrumentable.rb:31:in `instrument'
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/node.rb:616:in `logging'
    from /app/vendor/bundle/ruby/2.1.0/gems/newrelic_moped-0.0.11/lib/newrelic_moped/instrumentation.rb:40:in `block in logging_with_newrelic_trace'
    from /app/vendor/bundle/ruby/2.1.0/gems/newrelic_moped-0.0.11/lib/newrelic_moped/instrumentation.rb:59:in `call'
    from /app/vendor/bundle/ruby/2.1.0/gems/newrelic_moped-0.0.11/lib/newrelic_moped/instrumentation.rb:59:in `logging_with_newrelic_trace'
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/node.rb:587:in `flush'
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/node.rb:391:in `process'
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/operation/read.rb:48:in `execute'
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/node.rb:648:in `read'
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/node.rb:90:in `command'
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/database.rb:73:in `block in command'
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/cluster.rb:276:in `with_secondary'
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/read_preference/secondary_preferred.rb:45:in `block in with_node'
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/read_preference/selectable.rb:65:in `call'
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/read_preference/selectable.rb:65:in `with_retry'
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/read_preference/secondary_preferred.rb:43:in `with_node'
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/database.rb:72:in `command'
    from /app/vendor/bundle/ruby/2.1.0/gems/moped-2.0.1/lib/moped/indexes.rb:88:in `drop'
    from (irb):27
    from /app/vendor/bundle/ruby/2.1.0/gems/railties-4.0.12/lib/rails/commands/console.rb:90:in `start'
    from /app/vendor/bundle/ruby/2.1.0/gems/railties-4.0.12/lib/rails/commands/console.rb:9:in `start'
    from /app/vendor/bundle/ruby/2.1.0/gems/railties-4.0.12/lib/rails/commands.rb:62:in `<top (required)>'
    from /app/bin/rails:8:in `require'
    from /app/bin/rails:8:in `<main>'

[SFEley]

:+1: - we're hitting this too. From diving into the code, it looks like the database.command method always runs according to the configured 'read' preference — which means that if you configure reads to hit secondary nodes, any DB command you run (like, say, dropping indexes) will also run on secondary nodes. This is nuts, as the vast majority of database commands should be hitting the primary.