Delete from specified index

[DanBrk]

In some cases docs in Elastic are not deleted and I get the following error: Failed to find unique document [theDocId] for deletion using index pattern *

After enabling and looking at the Trace logs, I can see that the doc with same ID appears in two collections. The delete operation is ignored in this case since hits.total.value == 2 . This is how the code is implemented.

I would like to delete the doc from the same index that match the source collection namespace. For example, if doc has been deleted from collection DB1.Collection1, I would like it to be searched and deleted only from index db1.collection1

Is there a way to make the delete operation be like that ?

[rwynn]

You can actually remove that 1 result restriction with

disable-delete-protection = true

It would then delete any document that matches the search and not restrict it to 1 match.

The other setting you have in this regard is delete-index-pattern. That index pattern determines the scope of the indexes searched for deleted documents.

The reason this is difficult is that monstache doesn't really know where you have ended up putting the document, since it can be rerouted by middleware.

[rwynn]

There could be another strategy added like you mention to specifically target index a.b if the document was deleted from colleciton a.b. But that does not exist yet. I think probably because many people opt to rename the index that they store the documents to and search was the most general solution.

[DanBrk]

There is also an option delete-strategy=2 that completely ignore document deletes in MongoDB. I am using JavaScript middleware. Can I delete that from the transform script?

[rwynn]

I don't believe a delete in MongoDB passes through the javascript function.

Only if you update in MongoDB you can return null from the javascript middleware and it would attempt to delete that document. That was to handle a scenario where you did a soft delete in MongoDB but wanted to remove the document in Elasticsearch.