haystack-core-integrations
haystack-core-integrations copied to clipboard
Standardize streaming and function calling (FC) compatibility across all generators
Currently, FC is not supported when streaming is enabled. This limitation affects all generators using streaming capabilities.
Proposed Solutions:
-
Enable streaming for FC across all generators:
- Implement function calling to work with streaming in all generators
- Ensure thorough testing
-
Disable streaming for FC across all generators:
- Automatically disable streaming when function calling is required
- Provide clear documentation explaining why streaming is disabled for function calling, outlining the technical limitations and performance considerations
Action Required:
- Decide whether to enable streaming for function calling or to disable it and update generators accordingly
- Update documentation to reflect the changes and provide necessary explanations for users
Documentation Update:
- Include a section in the documentation explaining the incompatibility between streaming and function calling
- Detail the reasons for disabling streaming when function calling is enabled focusing on technical constraints