Julian Risch

feat: add batch evaluation method for pipelines

2

**Related Issue(s)**: closes #2636 **Proposed changes**: - Add a `pipeline.eval_batch` method - Add a `_build_eval_dataframe_from_batches` method that calls `_build_eval_dataframe` internally - I went for this solution to keep code duplication...

### Related Issues - As part of the support of elasticsearch 8, this PR is related to https://github.com/deepset-ai/haystack/issues/2810 ### Proposed Changes: ### How did you test it? ### Notes for...

Pin elasticsearch to version 8.3.3 for testing

1

### Related Issues - As part of the support of elasticsearch 8, this PR is related to https://github.com/deepset-ai/haystack/issues/2810 ### Proposed Changes: ### How did you test it? ### Notes for...

**Describe the bug** When executing a `TranslationWrapperPipeline` (for example as in the test case `test_extractive_qa_eval_translation`) with `run_batch` with the `debug=True` setting, the format of the `queries` passed to the retriever...

Incomplete debug results for pipeline with two retrievers

1

**Describe the bug** If a pipeline contains two retrievers then the debug results only contain the results of one of the two retrievers if the pipeline is executed with the...

To ease transition of users from OSS to DC, we would like to align their REST APIs. **Open Questions** * This might be a re-occurring topic. I am wondering what...

Pipeline eval should use batch processing for queries

1

We recently added a `run_batch()` method in every pipeline node, which can increase inference speed when batching multiple queries: https://github.com/deepset-ai/haystack/pull/2481 However, `pipeline.eval()` internally still calls `pipeline.run()` and evaluation is slow....

no_answer_score scaling is redundant

1

Currently, the no_answer scores use an `expit(score/8)` function to be scaled to the interval 0 to 1. However, the score should be between 0 and 1 even before that because...

**Is your feature request related to a problem? Please describe.** There was a new pylint release https://pylint.pycqa.org/en/latest/whatsnew/2/2.15/index.html which added a new checker missing-timeout. We disabled the check for the respective...

**Is your feature request related to a problem? Please describe.** Creating a PyPI release involves the same list of steps every time we have a new release. Manually going through...