elasticsearch icon indicating copy to clipboard operation
elasticsearch copied to clipboard

Add the parameter `failure_store` to multi-target syntax APIs

Open gmarouli opened this issue 1 year ago • 3 comments

In this PR we introduce the failure_store query param in all the APIs that are using org.elasticsearch.action.support.IndicesOptions#fromRequest and we do our best to use these values in the org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.

The param's name is failure_store and can take one of the three values:

  • false, meaning we do not include the failure indices, (default in most cases)
  • true, meaning we do include the failure indices
  • only, meaning we only include failure indices and not regular indices

Furthermore, we introduce one more flag allowFailureIndices which is not configurable by a user and it allows us to reject a failure index if the action does not support it. This works just like the closed indices.

The benefit of this approach is that we stay inline with how things currently work and it applies to requests of concrete failure indices or data streams.

Follow up work:

  • [ ] Set the allowFailureIndices to false when an action does not support it like we do in the org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest.
  • [ ] Test more thoroughly.
  • [ ] Test if it works correctly with security enabled.

gmarouli avatar Feb 12 '24 10:02 gmarouli

Pinging @elastic/es-data-management (Team:Data Management)

elasticsearchmachine avatar Feb 13 '24 13:02 elasticsearchmachine

This is looking really good! There's a good number of changes so I left a few small comments to start and will get the rest of the review completed today.

This morning, I extracted some changes in a different PR: https://github.com/elastic/elasticsearch/pull/105535. I thought it might make it easier, if it's not, feel free to ignore :p

gmarouli avatar Feb 15 '24 13:02 gmarouli

This morning, I extracted some changes in a different PR: #105535. I thought it might make it easier, if it's not, feel free to ignore :p

Thanks for splitting that out, but I think I have already looked over all those files in the original. 😅

The last I need to look through on this is the actual resolution logic, so we can probably close the other PR out and just finish the review process on this PR.

jbaiera avatar Feb 15 '24 17:02 jbaiera

@elasticmachine update branch

gmarouli avatar Mar 05 '24 13:03 gmarouli

This was a complicated one due to how much surface area it affects. Great work on this!

Thank you for the thorough review Jimmy, that wasn't easy!

Btw, I added one test that I had forgotten after the review but it's a small one so I am guessing you would be ok with it. If you have any objections let me know.

gmarouli avatar Mar 06 '24 08:03 gmarouli

@elasticmachine update branch

gmarouli avatar Mar 06 '24 09:03 gmarouli

FYI, I caused a merge mess when I forgot to pull before pushing, I used hard reset and force push to go back to a clean state and then fix it. No meaningful code was involved in this process.

gmarouli avatar Mar 06 '24 09:03 gmarouli