Add Bulk Item Failure Count

[mgodwan]

Is your feature request related to a problem? Please describe.

• Opensearch returns a 200 error response for a Bulk call, even though there can be partial failures within the complete request.
• Today, when an operator wants to gather data on the number of bulk item failures seen on a node, there is no way to obtain the data directly from the node/cluster.
• There are cases we have seen where the clients may not be fully aware of the stats for partial failures in bulk API.

Describe the solution you'd like

• Add a stat within the indexing actions (at the coordinating node) which will count the number of errors of each type.
• These actions can publish a counter regularly for each error type (400/429/500, etc.) while handling indexing requests.
• The counter will be exposed using an API which can be used by clients to query (e.g. using node stats API) the number of item level failures. The counter will also be the building block to be consumed through the PA plugin or any additional plugins/components.
• Based on the way Opensearch publishes all cumulative stats, most of the clients already setup a regular poller which allows them to make sense of the cumulative counter based on the last value seen for a node and the current value seen for a node to gather any insights over a desired time period. This will add value to all those use cases.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.