haystack icon indicating copy to clipboard operation
haystack copied to clipboard

feat: Add default OutputAdapter filters (post 2.0)

Open vblagoje opened this issue 1 year ago • 1 comments

Why:

Let's keep this PR open and continue collecting useful filters that keep repeating. Whenever you find yourself reusing some filter over and over again - add it here.

With these filters, developers can seamlessly adjust values being passed between components, ensuring the Haystack components receive data in suitable formats.

The first batch of standard filters includes:

  • 'change_role' - adjusting chat message roles
  • 'prepare_fc_params' - preparing function call parameters
  • 'tojson' - converting JSON strings to Python dictionaries.

What:

The pull request introduces the output_adapter_filters.py module that offers a set of custom filters for the OutputAdapter. The ALL_FILTERS dictionary contains these newly-defined filters and is included in test_output_adapter.py.

How can it be used:

The filters can be employed in pipelines by adding custom filters to the OutputAdapter instance in custom_filters. All filters can be used simultaneously or selectively. When an OutputAdapter instance is serialized with custom filters, the serialization includes custom filter definitions, ensuring the filters' persistency across various execution environments.

How did you test it:

Unit tests were executed to examine the serialization and deserialization of an OutputAdapter instance with the ALL_FILTERS dictionary added as custom filters. Assertions validate the custom filters' existence in both the serialized and deserialized OutputAdapter instances.

Notes for the reviewer:

When utilizing filters in production environments, sanitize and validate inputs carefully, especially if data originates from untrusted sources, to minimize potential security concerns.

vblagoje avatar Feb 23 '24 15:02 vblagoje

Pull Request Test Coverage Report for Build 8171735291

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-0.06%) to 89.98%

Totals Coverage Status
Change from base Build 8156947262: -0.06%
Covered Lines: 5298
Relevant Lines: 5888

💛 - Coveralls

coveralls avatar Feb 23 '24 15:02 coveralls