haystack icon indicating copy to clipboard operation
haystack copied to clipboard

Add Distribution based rank fusion mode

Open nickprock opened this issue 8 months ago • 7 comments

Related Issues

  • fixes #7914

Proposed Changes:

How it works:

from haystack import Document
from haystack.components.joiners.document_joiner import DocumentJoiner

joiner = DocumentJoiner(join_mode="distribution_based_rank_fusion")
        documents_1 = [
            Document(content="a", score=0.6),
            Document(content="b", score=0.2),
            Document(content="c", score=0.5),
        ]
        documents_2 = [
            Document(content="d", score=0.5),
            Document(content="e", score=0.8),
            Document(content="f", score=1.1, meta={"key": "value"}),
            Document(content="g", score=0.3),
            Document(content="a", score=0.3),
        ]
        output = joiner.run([documents_1, documents_2])

# output
# [
#               Document(content="a", score=0.66),
#               Document(content="b", score=0.27),
#               Document(content="c", score=0.56),
#               Document(content="d", score=0.44),
#               Document(content="e", score=0.60),
#               Document(content="f", score=0.76, meta={"key": "value"}),
#               Document(content="g", score=0.33),
#  ]

How did you test it?

unit tests

Checklist

nickprock avatar Jun 23 '24 15:06 nickprock