Add "delete by query" support

[romanbsd]

Tire doesn't currently support this feature: http://www.elasticsearch.org/guide/reference/api/delete-by-query.html

I hacked together the following solution (we're using it in production). Unfortunately I won't have time to work on this further.

https://gist.github.com/2346772

[mereghost]

Hello @karmi, any plans on supporting delete by query? If there is, I can work on the implementation and submit a pull request.

Cheers!

[thoughtpunch]

Any word on this? Need to remove items from the search index that have been soft-deleted via ActsAsParanoid. Would like to stay withing the Tire API if posssible, something like Tire.index("users").remove_from_index("5243")

[veesahni]

@thoughtpunch simple use cases can be handled like this:

Tire::Configuration.client.delete "#{Tire::Configuration.url}/twitter/tweets/_query?q=a:23"

[karmi]

Notice it's easy to use Tire's API/DSL for manually invoking the "Delete by Query" API, until there's a proper support:

# Dangerous code removed

See the code below

[Papipo]

In order to setup my tests, I want to delete all documents across al indexes, I tried using this:

index = Tire::Index.new('_all')

query = Tire::Search::Search.new do
  query { all }
end

Tire::Configuration.client.delete [index.url, query.to_hash.to_param].join('?')

But I can't search anymore. Does this delete indexes somehow?

[karmi]

@Papipo Yes

[Papipo]

Well, now I am using something simpler:

Tire::Configuration.client.delete "#{Tire::Configuration.url}/_all"

My problem was the async nature of the setup, so now I am calling refresh() after save and destroy.

[Will-Sommers]

@karmi any news on this now?

[jurgens]

@karmi could you please explain why index gets deleted on "manually invoking the "Delete by Query" API"? (see comment above - https://github.com/karmi/tire/issues/309#issuecomment-9699113)

[ches]

@jurgens @karmi's example is constructing a DELETE request to the resource endpoint of an index, e.g. http://localhost:9200/twitter/, plus some parameters. If query.to_hash.to_param evaluates to an empty string, which you'll find in @Papipo's query that it does, then you're effectively making the administrative delete index request that @karmi linked to.

[jurgens]

@ches thank you

[karmi]

@jurgens @Will-Sommers @thoughtpunch Sorry, the advice I gave was incorrect and dangerous (@ches above is right), I apologize.

This would be the workaround for the moment:

require 'tire'

index = Tire::Index.new('articles')

index = Tire::Index.new('articles') do
  delete
  store title: 'x'
  store title: 'y'
  store title: 'z'
  refresh
end

query = Tire::Search::Search.new do
  query { term :title, 'x' }
end

p query.to_hash

puts query.to_curl

puts "curl '#{index.url}/_search?source=#{Tire::Utils.escape(query.to_hash.to_json)}'"

puts "curl -X DELETE '#{index.url}/_query?source=#{Tire::Utils.escape(query.to_hash[:query].to_json)}'"

p Tire::Configuration.client.delete "#{index.url}/_query?source=#{Tire::Utils.escape(query.to_hash[:query].to_json)}"

---

NOTE: Eg. the Curb gem still doesn't support HTTP bodies with DELETE, making the support a bit cumbersome.

[jurgens]

Thank you @karmi that was helpful!

[phoet]

is this similar to #771?

[karmi]

@phoet Yes -- we could support delete_by_query by serializing the payload into the source URL parameter.

[phoet]

@karmi would you consider this a "hack"? and prefer the extension to RestClient for DELETE with payload?

[karmi]

@phoet I think that would qualify as an "acceptable" hack :)

(Of course provided we have unit+integration tests, etc)

[ches]

I believe 55d12c0caaf09f80 closed this.

[HenleyChiu]

@karmi I recommend deleting your post way up above if it's dangerous. Someone in a rush might just execute what you suggested w/o scrolling down below.

[x0bandeira]

There are a couple of fixes to make this work, mainly the curl logging on failure and the json for query on ?source=. Please review https://github.com/karmi/retire/pull/964