Model Callbacks create ES dependency?

[colllin]

Thank you for all of your awesome work on this project.

My ES provider (yeah, yeah, I'm lazy...) had an outage today that took down major parts of my application. Am I doing something to create a dependency on the ES service, or is there a dependency somewhere in the Tire callbacks? I think what's happening is that it's trying to update the index when I save my model, which is failing / timing out in some way, which is timing out the entire request.

[nz]

Relevant to my interests.

If I recall correctly, the mapping method defined in the Tire::Model::Indexing module will attempt to make a request to the Elasticsearch server when called. For most apps, as per the examples in the documentation, this method is run during the evaluation of the class itself, i.e., during the application boot.

If not able to reach the ES server at this time, the method will throw an exception causing the application to fail to boot.

https://github.com/karmi/retire/blob/master/lib/tire/model/indexing.rb#L30-65

A good place to start would be to wrap the typical call to mapper with a rescue block that catches and logs the exception:

class Article
  # ...
  begin
    mapping do
      indexes :id,    :index    => :not_analyzed
      indexes :title, :analyzer => 'snowball', :boost => 100
      # ...
    end
  rescue Exception => e
    logger.error "Failed to check/set index mapping during boot: #{e.message}"
  end
end

(Is that all correct and up to date? Been a while since I've read through this code.)

trying to update the index when I save my model

That's definitely a common one — an after_save hook which calls a method to post an update back to ES. This is also relevant to some other issues, like #639. Use of after_commit (vs. after_save) is also mentioned in the main Tire Readme, though I'm not immediately sure how that relates to existing use of after_save. Reading through the modules you're including to your model should clear that up a bit more.

There's also another project, tire_async_index, which looks to extend Tire with Resque and Sidekiq support to process updates asynchronously.

[nz]

Followup, regarding "Model Callbacks" by which I suppose you mean Tire::Model::Callbacks.

If those tire.update_index methods throw an exception, then creating and updating records will fail. This could be dealt with by rescuing the relevant exception in your controller actions, and either retrying or presenting a graceful error message.

The best option is to use an approach which queues updates to be processed in a batch when the server is back on. That does seem to be a little outside of Tire's scope, so I'd be interested to learn how other people are doing that.