haystack
haystack copied to clipboard
Add Distribution based rank fusion mode
Related Issues
- fixes #7914
Proposed Changes:
How it works:
from haystack import Document
from haystack.components.joiners.document_joiner import DocumentJoiner
joiner = DocumentJoiner(join_mode="distribution_based_rank_fusion")
documents_1 = [
Document(content="a", score=0.6),
Document(content="b", score=0.2),
Document(content="c", score=0.5),
]
documents_2 = [
Document(content="d", score=0.5),
Document(content="e", score=0.8),
Document(content="f", score=1.1, meta={"key": "value"}),
Document(content="g", score=0.3),
Document(content="a", score=0.3),
]
output = joiner.run([documents_1, documents_2])
# output
# [
# Document(content="a", score=0.66),
# Document(content="b", score=0.27),
# Document(content="c", score=0.56),
# Document(content="d", score=0.44),
# Document(content="e", score=0.60),
# Document(content="f", score=0.76, meta={"key": "value"}),
# Document(content="g", score=0.33),
# ]
How did you test it?
unit tests
Checklist
- I have read the contributors guidelines and the code of conduct
- I have updated the related issue with new insights and changes
- I added unit tests and updated the docstrings
- I've used one of the conventional commit types for my PR title:
fix:
,feat:
,build:
,chore:
,ci:
,docs:
,style:
,refactor:
,perf:
,test:
. - I documented my code
- I ran pre-commit hooks and fixed any issue