haystack
haystack copied to clipboard
feat: Add default OutputAdapter filters (post 2.0)
Why:
Let's keep this PR open and continue collecting useful filters that keep repeating. Whenever you find yourself reusing some filter over and over again - add it here.
With these filters, developers can seamlessly adjust values being passed between components, ensuring the Haystack components receive data in suitable formats.
The first batch of standard filters includes:
- 'change_role' - adjusting chat message roles
- 'prepare_fc_params' - preparing function call parameters
- 'tojson' - converting JSON strings to Python dictionaries.
What:
The pull request introduces the output_adapter_filters.py
module that offers a set of custom filters for the OutputAdapter
. The ALL_FILTERS
dictionary contains these newly-defined filters and is included in test_output_adapter.py
.
How can it be used:
The filters can be employed in pipelines by adding custom filters to the OutputAdapter
instance in custom_filters
. All filters can be used simultaneously or selectively. When an OutputAdapter
instance is serialized with custom filters, the serialization includes custom filter definitions, ensuring the filters' persistency across various execution environments.
How did you test it:
Unit tests were executed to examine the serialization and deserialization of an OutputAdapter
instance with the ALL_FILTERS
dictionary added as custom filters. Assertions validate the custom filters' existence in both the serialized and deserialized OutputAdapter
instances.
Notes for the reviewer:
When utilizing filters in production environments, sanitize and validate inputs carefully, especially if data originates from untrusted sources, to minimize potential security concerns.
Pull Request Test Coverage Report for Build 8171735291
Details
- 0 of 0 changed or added relevant lines in 0 files are covered.
- No unchanged relevant lines lost coverage.
- Overall coverage decreased (-0.06%) to 89.98%
Totals | |
---|---|
Change from base Build 8156947262: | -0.06% |
Covered Lines: | 5298 |
Relevant Lines: | 5888 |