Adding support for explain in rrf

[pmpailis]

In this PR we add support for explaining results and scores in RRF queries. To do this, this PR makes use of the ExplainPhase fetch sub-phase that is executed on the shards when explain=true. To achieve this a new explainHit method is specified for each ranker, so that they take a RankDoc and provide explanations on how the final score and ranks have been computed.

Since this takes place on the shards, we also have to serialize and pass the RankDoc objects that are associated with each shard during the FetchSearchPhase execution. This is enabled iff both explain and a RankBuilder are defined in the request, otherwise there is no impact at all (i.e. a null object is to be serialized and sent).

We also aim to match the queries executed through the RankBuilder with any _name associated with them, so that the output can be easier to interpret (if no _name is provided for a query, then we just make use of the query's index when executed).

Example RRF explain output:

{
    "hits":
    [
        {
            "_index": "example-index",
            "_id": "3",
            "_score": 0.8333334,
            "_rank": 1,
            "_explanation":
            {
                "value": 0.8333334,                                                                                                                                               
                "description": "rrf score: [0.8333334] computed for initial ranks [2, 1] with rankConstant: [1] as sum of [1 / (rank + rankConstant)] for each query",            
                "details":                                                                                                                                                        
                [
                    {
                        "value": 2,                                                                                                                                               
                        "description": "rrf score: [0.33333334], for rank [2] in query at index [0] computed as [1 / (2 + 1]), for matching query with score: ",
                        "details":                                                                                                                                                
                        [
                            {
                                "value": 0.15876243,
                                "description": "weight(text:rrf in 0) [PerFieldSimilarity], result of:",
                                "details":
                                [
                                    ...
                                ]
                            }
                        ]
                    },
                    {
                        "value": 1,                                                                                                                                              
                        "description": "rrf score: [0.5], for rank [1] in query ["my_knn_query"] computed as [1 / (1 + 1]), for matching query with score: ",
                        "details":
                        [
                            {
                                "value": 1,
                                "description": "within top k documents",
                                "details":
                                []
                            }
                        ]
                    }
                ]
            }
        }
        ...
    ]
}

[github-actions[bot]]

Documentation preview:

• ✨ Changed pages

[elasticsearchmachine]

Pinging @elastic/es-search (Team:Search)

[elasticsearchmachine]

Hi @pmpailis, I've created a changelog YAML for you.

[elasticsearchmachine]

Hi @pmpailis, I've updated the changelog YAML for you.

[pmpailis]

@elasticmachine update branch

[pmpailis]

@elasticmachine update branch

[pmpailis]

@elasticmachine update branch

[pmpailis]

@elasticmachine update branch